2026-02-13T10:31:48.067 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-13T10:31:48.085 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T10:31:48.182 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841 branch: wip-jcollin-testing-20260212.151113-tentacle description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: jcollin@redhat.com first_in_suite: false flavor: default job_id: '48841' kernel: sha1: distro last_in_suite: false machine_type: trial meta: - desc: 'setup ceph/squid ' name: jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-jcollin-testing-20260212.151113-tentacle ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: mon pg warn min per osd: 0 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - is offline because no MDS - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - BLUESTORE_SLOW_OP_ALERT - slow operation indications in BlueStore - experiencing slow operations in BlueStore - MGR_MODULE_ERROR - OSD_DOWN - osd.* is down - OSD_DOWN - osd.*is down - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - osds down - osd.* is down - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - pg .* is stuck inactive - pg .* is .*degraded - pg .* is stuck peering sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-jcollin-testing-20260212.151113-tentacle sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 owner: scheduled_jcollin@soko04.front.sepia.ceph.com priority: 45 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 5413 sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 sleep_before_teardown: 0 subset: 425/2048 suite: fs suite_branch: wip-jcollin-testing-20260212.151113-tentacle suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 targets: trial121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBINSLVWM5s+zEd11abT84k80Z+nBM3Rh+maXac0llKSWZfDwEDXio0ev46EBN3N5DCcFeI5OGUrlL6EDMLq7HhU= trial163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCSrTbKnSm9PGke2LBxcKrJH012X1CYDAOsQKjRIsYB3ifLlDP8kMwkQn9lPjRO4ISlp7Tf2Se+f4dYcRVZbjx8= tasks: - install: branch: squid exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: squid conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:squid roleless: true - print: '**** done end installing squid cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - ceph osd require-osd-release tentacle - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /tank/home/jcollin/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: - "local branch = yaml.kernel.branch\nif branch and not yaml.kernel.branch:find\ \ \"-all$\" then\n log.debug(\"removing default kernel specification: %s\", yaml.kernel)\n\ \ py_attrgetter(yaml.kernel).pop('branch', nil)\n py_attrgetter(yaml.kernel).pop('deb',\ \ nil)\n py_attrgetter(yaml.kernel).pop('flavor', nil)\n py_attrgetter(yaml.kernel).pop('kdb',\ \ nil)\n py_attrgetter(yaml.kernel).pop('koji', nil)\n py_attrgetter(yaml.kernel).pop('koji_task',\ \ nil)\n py_attrgetter(yaml.kernel).pop('rpm', nil)\n py_attrgetter(yaml.kernel).pop('sha1',\ \ nil)\n py_attrgetter(yaml.kernel).pop('tag', nil)\nend\n" variables: fail_fs: false teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-13_09:34:52 tube: trial upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: jcollin verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2026-02-13T10:31:48.182 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa; will attempt to use it 2026-02-13T10:31:48.182 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa/tasks 2026-02-13T10:31:48.182 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-13T10:31:48.184 INFO:teuthology.task.internal:Checking packages... 2026-02-13T10:31:48.184 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '79a8b77b79a4b70f9c11cf81d24d449d40916c58' 2026-02-13T10:31:48.184 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T10:31:48.184 INFO:teuthology.packaging:ref: None 2026-02-13T10:31:48.184 INFO:teuthology.packaging:tag: None 2026-02-13T10:31:48.184 INFO:teuthology.packaging:branch: wip-jcollin-testing-20260212.151113-tentacle 2026-02-13T10:31:48.184 INFO:teuthology.packaging:sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:31:48.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-jcollin-testing-20260212.151113-tentacle 2026-02-13T10:31:48.306 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-667.g79a8b77b 2026-02-13T10:31:48.308 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-13T10:31:48.319 INFO:teuthology.task.internal:no buildpackages task found 2026-02-13T10:31:48.319 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-13T10:31:48.331 INFO:teuthology.task.internal:Saving configuration 2026-02-13T10:31:48.349 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-13T10:31:48.362 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-13T10:31:48.436 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial121.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 10:28:32.371362', 'locked_by': 'scheduled_jcollin@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBINSLVWM5s+zEd11abT84k80Z+nBM3Rh+maXac0llKSWZfDwEDXio0ev46EBN3N5DCcFeI5OGUrlL6EDMLq7HhU='} 2026-02-13T10:31:48.490 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial163.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 10:28:32.370737', 'locked_by': 'scheduled_jcollin@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCSrTbKnSm9PGke2LBxcKrJH012X1CYDAOsQKjRIsYB3ifLlDP8kMwkQn9lPjRO4ISlp7Tf2Se+f4dYcRVZbjx8='} 2026-02-13T10:31:48.490 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-13T10:31:48.503 INFO:teuthology.task.internal:roles: ubuntu@trial121.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2026-02-13T10:31:48.503 INFO:teuthology.task.internal:roles: ubuntu@trial163.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2026-02-13T10:31:48.503 INFO:teuthology.run_tasks:Running task console_log... 2026-02-13T10:31:48.637 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1581badbc0>, signals=[15]) 2026-02-13T10:31:48.637 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-13T10:31:48.648 INFO:teuthology.task.internal:Opening connections... 2026-02-13T10:31:48.648 DEBUG:teuthology.task.internal:connecting to ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:31:48.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:31:48.712 DEBUG:teuthology.task.internal:connecting to ubuntu@trial163.front.sepia.ceph.com 2026-02-13T10:31:48.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:31:48.774 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-13T10:31:48.786 DEBUG:teuthology.orchestra.run.trial121:> uname -m 2026-02-13T10:31:48.800 INFO:teuthology.orchestra.run.trial121.stdout:x86_64 2026-02-13T10:31:48.800 DEBUG:teuthology.orchestra.run.trial121:> cat /etc/os-release 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:NAME="CentOS Stream" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:VERSION="9" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:ID="centos" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:ID_LIKE="rhel fedora" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:VERSION_ID="9" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:PLATFORM_ID="platform:el9" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:ANSI_COLOR="0;31" 2026-02-13T10:31:48.857 INFO:teuthology.orchestra.run.trial121.stdout:LOGO="fedora-logo-icon" 2026-02-13T10:31:48.858 INFO:teuthology.orchestra.run.trial121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T10:31:48.858 INFO:teuthology.orchestra.run.trial121.stdout:HOME_URL="https://centos.org/" 2026-02-13T10:31:48.858 INFO:teuthology.orchestra.run.trial121.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T10:31:48.858 INFO:teuthology.orchestra.run.trial121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T10:31:48.858 INFO:teuthology.orchestra.run.trial121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T10:31:48.858 INFO:teuthology.lock.ops:Updating trial121.front.sepia.ceph.com on lock server 2026-02-13T10:31:48.934 DEBUG:teuthology.orchestra.run.trial163:> uname -m 2026-02-13T10:31:48.949 INFO:teuthology.orchestra.run.trial163.stdout:x86_64 2026-02-13T10:31:48.950 DEBUG:teuthology.orchestra.run.trial163:> cat /etc/os-release 2026-02-13T10:31:49.006 INFO:teuthology.orchestra.run.trial163.stdout:NAME="CentOS Stream" 2026-02-13T10:31:49.006 INFO:teuthology.orchestra.run.trial163.stdout:VERSION="9" 2026-02-13T10:31:49.006 INFO:teuthology.orchestra.run.trial163.stdout:ID="centos" 2026-02-13T10:31:49.006 INFO:teuthology.orchestra.run.trial163.stdout:ID_LIKE="rhel fedora" 2026-02-13T10:31:49.006 INFO:teuthology.orchestra.run.trial163.stdout:VERSION_ID="9" 2026-02-13T10:31:49.006 INFO:teuthology.orchestra.run.trial163.stdout:PLATFORM_ID="platform:el9" 2026-02-13T10:31:49.006 INFO:teuthology.orchestra.run.trial163.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T10:31:49.007 INFO:teuthology.orchestra.run.trial163.stdout:ANSI_COLOR="0;31" 2026-02-13T10:31:49.007 INFO:teuthology.orchestra.run.trial163.stdout:LOGO="fedora-logo-icon" 2026-02-13T10:31:49.007 INFO:teuthology.orchestra.run.trial163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T10:31:49.007 INFO:teuthology.orchestra.run.trial163.stdout:HOME_URL="https://centos.org/" 2026-02-13T10:31:49.007 INFO:teuthology.orchestra.run.trial163.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T10:31:49.007 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T10:31:49.007 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T10:31:49.007 INFO:teuthology.lock.ops:Updating trial163.front.sepia.ceph.com on lock server 2026-02-13T10:31:49.088 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-13T10:31:49.100 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-13T10:31:49.111 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-13T10:31:49.111 DEBUG:teuthology.orchestra.run.trial121:> test '!' -e /home/ubuntu/cephtest 2026-02-13T10:31:49.113 DEBUG:teuthology.orchestra.run.trial163:> test '!' -e /home/ubuntu/cephtest 2026-02-13T10:31:49.127 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-13T10:31:49.138 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-13T10:31:49.138 DEBUG:teuthology.orchestra.run.trial121:> test -z $(ls -A /var/lib/ceph) 2026-02-13T10:31:49.168 DEBUG:teuthology.orchestra.run.trial163:> test -z $(ls -A /var/lib/ceph) 2026-02-13T10:31:49.185 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-13T10:31:49.343 INFO:teuthology.run_tasks:Running task kernel... 2026-02-13T10:31:49.362 INFO:teuthology.task.kernel:normalize config orig: {'sha1': 'distro'} 2026-02-13T10:31:49.362 INFO:teuthology.task.kernel:config {'host.a': {'sha1': 'distro'}, 'host.b': {'sha1': 'distro'}}, timeout 300 2026-02-13T10:31:49.362 DEBUG:teuthology.orchestra.run.trial121:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T10:31:49.362 DEBUG:teuthology.orchestra.run.trial163:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T10:31:49.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:31:49.377 DEBUG:teuthology.orchestra.run.trial163:> uname -r 2026-02-13T10:31:49.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:31:49.378 DEBUG:teuthology.orchestra.run.trial121:> uname -r 2026-02-13T10:31:49.433 INFO:teuthology.orchestra.run.trial163.stdout:5.14.0-645.el9.x86_64 2026-02-13T10:31:49.433 INFO:teuthology.task.kernel:Running kernel on trial163: 5.14.0-645.el9.x86_64 2026-02-13T10:31:49.433 DEBUG:teuthology.orchestra.run.trial163:> sudo yum install -y kernel 2026-02-13T10:31:49.433 INFO:teuthology.orchestra.run.trial121.stdout:5.14.0-645.el9.x86_64 2026-02-13T10:31:49.434 INFO:teuthology.task.kernel:Running kernel on trial121: 5.14.0-645.el9.x86_64 2026-02-13T10:31:49.434 DEBUG:teuthology.orchestra.run.trial121:> sudo yum install -y kernel 2026-02-13T10:31:50.622 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-13T10:31:51.791 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.9 MB 00:01 2026-02-13T10:31:53.660 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-13T10:31:54.965 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-13T10:31:56.979 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 7.8 MB 00:00 2026-02-13T10:31:57.889 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - Extras packages 101 kB/s | 20 kB 00:00 2026-02-13T10:31:58.384 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 7.8 MB 00:01 2026-02-13T10:31:58.829 INFO:teuthology.orchestra.run.trial121.stdout:Extra Packages for Enterprise Linux 23 MB/s | 20 MB 00:00 2026-02-13T10:31:59.307 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 20 kB 00:00 2026-02-13T10:32:00.246 INFO:teuthology.orchestra.run.trial163.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2026-02-13T10:32:01.980 INFO:teuthology.orchestra.run.trial121.stdout:lab-extras 2.0 MB/s | 50 kB 00:00 2026-02-13T10:32:03.042 INFO:teuthology.orchestra.run.trial121.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T10:32:03.062 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:32:03.063 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout: Package Architecture Version Repository Size 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:32:03.064 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:32:03.065 INFO:teuthology.orchestra.run.trial121.stdout:Install 4 Packages 2026-02-13T10:32:03.065 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:32:03.065 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 88 M 2026-02-13T10:32:03.065 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 135 M 2026-02-13T10:32:03.065 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:32:03.200 INFO:teuthology.orchestra.run.trial121.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 13 MB/s | 768 kB 00:00 2026-02-13T10:32:03.281 INFO:teuthology.orchestra.run.trial121.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 121 MB/s | 17 MB 00:00 2026-02-13T10:32:03.366 INFO:teuthology.orchestra.run.trial121.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 185 MB/s | 31 MB 00:00 2026-02-13T10:32:03.427 INFO:teuthology.orchestra.run.trial121.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 138 MB/s | 40 MB 00:00 2026-02-13T10:32:03.427 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:32:03.427 INFO:teuthology.orchestra.run.trial121.stdout:Total 243 MB/s | 88 MB 00:00 2026-02-13T10:32:03.475 INFO:teuthology.orchestra.run.trial163.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-13T10:32:03.614 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:32:03.821 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:32:03.821 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:32:04.166 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:32:04.167 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:32:04.530 INFO:teuthology.orchestra.run.trial163.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T10:32:04.550 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:32:04.551 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:32:04.551 INFO:teuthology.orchestra.run.trial163.stdout: Package Architecture Version Repository Size 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T10:32:04.552 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:32:04.553 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:32:04.553 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:32:04.553 INFO:teuthology.orchestra.run.trial163.stdout:Install 4 Packages 2026-02-13T10:32:04.553 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:32:04.553 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 88 M 2026-02-13T10:32:04.553 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 135 M 2026-02-13T10:32:04.553 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:32:05.262 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:32:05.545 INFO:teuthology.orchestra.run.trial121.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T10:32:05.564 INFO:teuthology.orchestra.run.trial121.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T10:32:06.036 INFO:teuthology.orchestra.run.trial163.stdout:(1/4): kernel-core-5.14.0-677.el9.x86_64.rpm 12 MB/s | 17 MB 00:01 2026-02-13T10:32:06.799 INFO:teuthology.orchestra.run.trial163.stdout:(2/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 18 MB/s | 40 MB 00:02 2026-02-13T10:32:06.843 INFO:teuthology.orchestra.run.trial163.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 38 MB/s | 31 MB 00:00 2026-02-13T10:32:07.045 INFO:teuthology.orchestra.run.trial163.stdout:(4/4): kernel-5.14.0-677.el9.x86_64.rpm 316 kB/s | 768 kB 00:02 2026-02-13T10:32:07.047 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:32:07.047 INFO:teuthology.orchestra.run.trial163.stdout:Total 35 MB/s | 88 MB 00:02 2026-02-13T10:32:07.235 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:32:07.341 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T10:32:07.370 INFO:teuthology.orchestra.run.trial121.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T10:32:07.437 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:32:07.437 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:32:07.781 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:32:07.781 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:32:08.869 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:32:09.152 INFO:teuthology.orchestra.run.trial163.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T10:32:09.171 INFO:teuthology.orchestra.run.trial163.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T10:32:10.397 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T10:32:10.403 INFO:teuthology.orchestra.run.trial121.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:10.945 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T10:32:10.974 INFO:teuthology.orchestra.run.trial163.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T10:32:13.502 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:14.022 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T10:32:14.028 INFO:teuthology.orchestra.run.trial163.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:17.130 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:44.072 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:44.094 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:44.160 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:44.161 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T10:32:44.161 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T10:32:44.161 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.213 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:32:44.214 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:32:44.273 DEBUG:teuthology.orchestra.run.trial121:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T10:32:44.299 INFO:teuthology.orchestra.run.trial121.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.300 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T10:32:44.300 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T10:32:44.300 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T10:32:44.300 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T10:32:44.300 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2026-02-13T10:32:44.300 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial121.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T10:32:44.301 DEBUG:teuthology.orchestra.run.trial121:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T10:32:44.364 INFO:teuthology.orchestra.run.trial121.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.365 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T10:32:44.365 DEBUG:teuthology.orchestra.run.trial121:> sudo rpm -qi grub2-tools 2026-02-13T10:32:44.433 INFO:teuthology.orchestra.run.trial121.stdout:Name : grub2-tools 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Epoch : 1 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Version : 2.06 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Release : 118.el9 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Architecture: x86_64 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Group : Unspecified 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Size : 8393909 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:License : GPLv3+ 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T10:32:44.434 INFO:teuthology.orchestra.run.trial121.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:Packager : builder@centos.org 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:Vendor : CentOS 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:Summary : Support tools for GRUB. 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:Description : 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T10:32:44.435 INFO:teuthology.orchestra.run.trial121.stdout:hardware devices. 2026-02-13T10:32:44.436 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:32:44.436 INFO:teuthology.orchestra.run.trial121.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T10:32:44.437 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T10:32:44.437 INFO:teuthology.task.kernel:Updating grub on trial121 to boot 5.14.0-677.el9.x86_64 2026-02-13T10:32:44.437 DEBUG:teuthology.orchestra.run.trial121:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T10:32:44.500 INFO:teuthology.orchestra.run.trial121.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T10:32:44.501 DEBUG:teuthology.orchestra.run.trial121:> sudo command -v grubby && echo yes || echo no 2026-02-13T10:32:44.564 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/grubby 2026-02-13T10:32:44.565 INFO:teuthology.orchestra.run.trial121.stdout:yes 2026-02-13T10:32:44.566 DEBUG:teuthology.orchestra.run.trial121:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T10:32:44.632 INFO:teuthology.orchestra.run.trial121.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T10:32:44.633 DEBUG:teuthology.orchestra.run.trial121:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.707 DEBUG:teuthology.orchestra.run.trial121:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T10:32:44.918 INFO:teuthology.orchestra.run.trial121.stderr:Generating grub configuration file ... 2026-02-13T10:32:44.981 INFO:teuthology.orchestra.run.trial121.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T10:32:44.988 INFO:teuthology.orchestra.run.trial121.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T10:32:45.240 INFO:teuthology.orchestra.run.trial121.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T10:32:45.247 INFO:teuthology.orchestra.run.trial121.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T10:32:45.279 INFO:teuthology.orchestra.run.trial121.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T10:32:45.286 INFO:teuthology.orchestra.run.trial121.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T10:32:45.309 INFO:teuthology.orchestra.run.trial121.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T10:32:45.315 INFO:teuthology.orchestra.run.trial121.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T10:32:45.397 INFO:teuthology.orchestra.run.trial121.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T10:32:45.405 INFO:teuthology.orchestra.run.trial121.stderr:done 2026-02-13T10:32:45.407 DEBUG:teuthology.orchestra.run.trial121:> test -d /sys/firmware/efi 2026-02-13T10:32:45.423 DEBUG:teuthology.orchestra.run.trial121:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T10:32:45.489 INFO:teuthology.orchestra.run.trial121.stdout:/boot/efi/EFI/BOOT 2026-02-13T10:32:45.489 INFO:teuthology.orchestra.run.trial121.stdout:/boot/efi/EFI/centos 2026-02-13T10:32:45.490 DEBUG:teuthology.orchestra.run.trial121:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T10:32:45.558 INFO:teuthology.orchestra.run.trial121.stdout:no 2026-02-13T10:32:45.558 DEBUG:teuthology.orchestra.run.trial121:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T10:32:45.622 INFO:teuthology.orchestra.run.trial121.stdout:yes 2026-02-13T10:32:45.623 DEBUG:teuthology.orchestra.run.trial121:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T10:32:45.689 DEBUG:teuthology.orchestra.run.trial121:> sudo shutdown -r now 2026-02-13T10:32:47.517 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:47.540 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:47.711 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:47.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T10:32:47.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T10:32:47.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T10:32:47.763 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T10:32:47.763 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:32:47.763 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:32:47.764 INFO:teuthology.orchestra.run.trial163.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T10:32:47.764 INFO:teuthology.orchestra.run.trial163.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T10:32:47.764 INFO:teuthology.orchestra.run.trial163.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T10:32:47.764 INFO:teuthology.orchestra.run.trial163.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T10:32:47.764 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:32:47.764 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:32:47.820 DEBUG:teuthology.orchestra.run.trial163:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T10:32:47.847 INFO:teuthology.orchestra.run.trial163.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T10:32:47.847 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T10:32:47.847 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T10:32:47.847 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T10:32:47.847 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T10:32:47.847 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2026-02-13T10:32:47.847 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial163.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T10:32:47.847 DEBUG:teuthology.orchestra.run.trial163:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T10:32:47.910 INFO:teuthology.orchestra.run.trial163.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T10:32:47.910 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T10:32:47.911 DEBUG:teuthology.orchestra.run.trial163:> sudo rpm -qi grub2-tools 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Name : grub2-tools 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Epoch : 1 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Version : 2.06 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Release : 118.el9 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Architecture: x86_64 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Group : Unspecified 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Size : 8393909 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:License : GPLv3+ 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Packager : builder@centos.org 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Vendor : CentOS 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Summary : Support tools for GRUB. 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:Description : 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:hardware devices. 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:32:47.979 INFO:teuthology.orchestra.run.trial163.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T10:32:47.980 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T10:32:47.980 INFO:teuthology.task.kernel:Updating grub on trial163 to boot 5.14.0-677.el9.x86_64 2026-02-13T10:32:47.980 DEBUG:teuthology.orchestra.run.trial163:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T10:32:48.045 INFO:teuthology.orchestra.run.trial163.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T10:32:48.046 DEBUG:teuthology.orchestra.run.trial163:> sudo command -v grubby && echo yes || echo no 2026-02-13T10:32:48.111 INFO:teuthology.orchestra.run.trial163.stdout:/sbin/grubby 2026-02-13T10:32:48.112 INFO:teuthology.orchestra.run.trial163.stdout:yes 2026-02-13T10:32:48.112 DEBUG:teuthology.orchestra.run.trial163:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T10:32:48.176 INFO:teuthology.orchestra.run.trial163.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T10:32:48.176 DEBUG:teuthology.orchestra.run.trial163:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T10:32:48.247 DEBUG:teuthology.orchestra.run.trial163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T10:32:48.455 INFO:teuthology.orchestra.run.trial163.stderr:Generating grub configuration file ... 2026-02-13T10:32:48.520 INFO:teuthology.orchestra.run.trial163.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T10:32:48.526 INFO:teuthology.orchestra.run.trial163.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T10:32:48.776 INFO:teuthology.orchestra.run.trial163.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T10:32:48.782 INFO:teuthology.orchestra.run.trial163.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T10:32:48.814 INFO:teuthology.orchestra.run.trial163.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T10:32:48.821 INFO:teuthology.orchestra.run.trial163.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T10:32:48.844 INFO:teuthology.orchestra.run.trial163.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T10:32:48.850 INFO:teuthology.orchestra.run.trial163.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T10:32:48.935 INFO:teuthology.orchestra.run.trial163.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T10:32:48.944 INFO:teuthology.orchestra.run.trial163.stderr:done 2026-02-13T10:32:48.946 DEBUG:teuthology.orchestra.run.trial163:> test -d /sys/firmware/efi 2026-02-13T10:32:48.962 DEBUG:teuthology.orchestra.run.trial163:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T10:32:49.026 INFO:teuthology.orchestra.run.trial163.stdout:/boot/efi/EFI/BOOT 2026-02-13T10:32:49.026 INFO:teuthology.orchestra.run.trial163.stdout:/boot/efi/EFI/centos 2026-02-13T10:32:49.027 DEBUG:teuthology.orchestra.run.trial163:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T10:32:49.092 INFO:teuthology.orchestra.run.trial163.stdout:no 2026-02-13T10:32:49.093 DEBUG:teuthology.orchestra.run.trial163:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T10:32:49.156 INFO:teuthology.orchestra.run.trial163.stdout:yes 2026-02-13T10:32:49.156 DEBUG:teuthology.orchestra.run.trial163:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T10:32:49.220 DEBUG:teuthology.orchestra.run.trial163:> sudo shutdown -r now 2026-02-13T10:32:49.293 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-13T10:33:15.758 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2026-02-13T10:33:15.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial121.front.sepia.ceph.com' 2026-02-13T10:33:15.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:33:19.266 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2026-02-13T10:33:19.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial163.front.sepia.ceph.com' 2026-02-13T10:33:19.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:34:15.797 DEBUG:teuthology.orchestra.remote:timed out 2026-02-13T10:34:19.272 DEBUG:teuthology.orchestra.remote:timed out 2026-02-13T10:34:24.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial121.front.sepia.ceph.com' 2026-02-13T10:34:24.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:34:24.938 DEBUG:teuthology.orchestra.run.trial121:> true 2026-02-13T10:34:25.103 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial121.front.sepia.ceph.com' 2026-02-13T10:34:25.103 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-677.el9.x86_64"... 2026-02-13T10:34:25.103 DEBUG:teuthology.orchestra.run.trial121:> uname -r 2026-02-13T10:34:25.159 INFO:teuthology.orchestra.run.trial121.stdout:5.14.0-677.el9.x86_64 2026-02-13T10:34:25.159 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T10:34:25.159 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T10:34:25.159 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T10:34:26.160 DEBUG:teuthology.parallel:result is None 2026-02-13T10:34:28.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial163.front.sepia.ceph.com' 2026-02-13T10:34:28.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:34:28.410 DEBUG:teuthology.orchestra.run.trial163:> true 2026-02-13T10:34:28.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial163.front.sepia.ceph.com' 2026-02-13T10:34:28.563 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-677.el9.x86_64"... 2026-02-13T10:34:28.563 DEBUG:teuthology.orchestra.run.trial163:> uname -r 2026-02-13T10:34:28.620 INFO:teuthology.orchestra.run.trial163.stdout:5.14.0-677.el9.x86_64 2026-02-13T10:34:28.621 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T10:34:28.621 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T10:34:28.621 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T10:34:29.623 DEBUG:teuthology.parallel:result is None 2026-02-13T10:34:29.623 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-13T10:34:29.634 INFO:teuthology.task.internal:Creating test directory... 2026-02-13T10:34:29.635 DEBUG:teuthology.orchestra.run.trial121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T10:34:29.637 DEBUG:teuthology.orchestra.run.trial163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T10:34:29.652 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-13T10:34:29.664 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-13T10:34:29.681 INFO:teuthology.task.internal:Creating archive directory... 2026-02-13T10:34:29.681 DEBUG:teuthology.orchestra.run.trial121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T10:34:29.693 DEBUG:teuthology.orchestra.run.trial163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T10:34:29.715 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-13T10:34:29.727 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-13T10:34:29.727 DEBUG:teuthology.orchestra.run.trial121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T10:34:29.750 DEBUG:teuthology.orchestra.run.trial163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T10:34:29.778 INFO:teuthology.orchestra.run.trial121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T10:34:29.779 INFO:teuthology.orchestra.run.trial163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T10:34:29.787 INFO:teuthology.orchestra.run.trial121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T10:34:29.788 INFO:teuthology.orchestra.run.trial163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T10:34:29.789 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-13T10:34:29.801 INFO:teuthology.task.internal:Configuring sudo... 2026-02-13T10:34:29.801 DEBUG:teuthology.orchestra.run.trial121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T10:34:29.831 DEBUG:teuthology.orchestra.run.trial163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T10:34:29.855 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-13T10:34:29.869 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-13T10:34:29.869 DEBUG:teuthology.orchestra.run.trial121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T10:34:29.896 DEBUG:teuthology.orchestra.run.trial163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T10:34:29.912 DEBUG:teuthology.orchestra.run.trial121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T10:34:30.060 DEBUG:teuthology.orchestra.run.trial121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T10:34:30.086 DEBUG:teuthology.orchestra.run.trial121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T10:34:30.232 DEBUG:teuthology.orchestra.run.trial121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T10:34:30.257 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:34:30.257 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T10:34:30.320 DEBUG:teuthology.orchestra.run.trial163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T10:34:30.420 DEBUG:teuthology.orchestra.run.trial163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T10:34:30.445 DEBUG:teuthology.orchestra.run.trial163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T10:34:30.569 DEBUG:teuthology.orchestra.run.trial163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T10:34:30.594 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:34:30.594 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T10:34:30.657 DEBUG:teuthology.orchestra.run.trial121:> sudo service rsyslog restart 2026-02-13T10:34:30.659 DEBUG:teuthology.orchestra.run.trial163:> sudo service rsyslog restart 2026-02-13T10:34:30.686 INFO:teuthology.orchestra.run.trial121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T10:34:30.725 INFO:teuthology.orchestra.run.trial163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T10:34:31.070 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-13T10:34:31.084 INFO:teuthology.task.internal:Starting timer... 2026-02-13T10:34:31.084 INFO:teuthology.run_tasks:Running task pcp... 2026-02-13T10:34:31.108 INFO:teuthology.run_tasks:Running task selinux... 2026-02-13T10:34:31.126 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2026-02-13T10:34:31.126 DEBUG:teuthology.orchestra.run.trial121:> sudo service auditd rotate 2026-02-13T10:34:31.156 INFO:teuthology.orchestra.run.trial121.stdout:Rotating logs: 2026-02-13T10:34:31.157 DEBUG:teuthology.orchestra.run.trial163:> sudo service auditd rotate 2026-02-13T10:34:31.187 INFO:teuthology.orchestra.run.trial163.stdout:Rotating logs: 2026-02-13T10:34:31.189 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-13T10:34:31.189 DEBUG:teuthology.orchestra.run.trial121:> /usr/sbin/getenforce 2026-02-13T10:34:31.214 INFO:teuthology.orchestra.run.trial121.stdout:Permissive 2026-02-13T10:34:31.214 DEBUG:teuthology.orchestra.run.trial163:> /usr/sbin/getenforce 2026-02-13T10:34:31.247 INFO:teuthology.orchestra.run.trial163.stdout:Permissive 2026-02-13T10:34:31.247 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial121.front.sepia.ceph.com': 'permissive', 'ubuntu@trial163.front.sepia.ceph.com': 'permissive'} 2026-02-13T10:34:31.247 DEBUG:teuthology.orchestra.run.trial121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-13T10:34:31.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:34:31.278 DEBUG:teuthology.orchestra.run.trial163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-13T10:34:31.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:34:31.312 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-13T10:34:31.312 DEBUG:teuthology.orchestra.run.trial121:> sudo /usr/sbin/setenforce permissive 2026-02-13T10:34:31.341 DEBUG:teuthology.orchestra.run.trial163:> sudo /usr/sbin/setenforce permissive 2026-02-13T10:34:31.377 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-13T10:34:31.393 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-13T10:34:31.395 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-13T10:34:31.527 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-13T10:34:31.538 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-13T10:34:31.539 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial121.front.sepia.ceph.com,trial163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-13T10:35:39.084 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial121.front.sepia.ceph.com'), Remote(name='ubuntu@trial163.front.sepia.ceph.com')] 2026-02-13T10:35:39.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial121.front.sepia.ceph.com' 2026-02-13T10:35:39.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:35:39.196 DEBUG:teuthology.orchestra.run.trial121:> true 2026-02-13T10:35:39.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial121.front.sepia.ceph.com' 2026-02-13T10:35:39.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial163.front.sepia.ceph.com' 2026-02-13T10:35:39.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T10:35:39.365 DEBUG:teuthology.orchestra.run.trial163:> true 2026-02-13T10:35:39.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial163.front.sepia.ceph.com' 2026-02-13T10:35:39.445 INFO:teuthology.run_tasks:Running task clock... 2026-02-13T10:35:39.460 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-13T10:35:39.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T10:35:39.461 DEBUG:teuthology.orchestra.run.trial121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T10:35:39.464 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T10:35:39.464 DEBUG:teuthology.orchestra.run.trial163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T10:35:39.490 INFO:teuthology.orchestra.run.trial121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T10:35:39.501 INFO:teuthology.orchestra.run.trial121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T10:35:39.512 INFO:teuthology.orchestra.run.trial163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T10:35:39.521 INFO:teuthology.orchestra.run.trial121.stderr:sudo: ntpd: command not found 2026-02-13T10:35:39.524 INFO:teuthology.orchestra.run.trial163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T10:35:39.530 INFO:teuthology.orchestra.run.trial121.stdout:506 Cannot talk to daemon 2026-02-13T10:35:39.543 INFO:teuthology.orchestra.run.trial121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T10:35:39.545 INFO:teuthology.orchestra.run.trial163.stderr:sudo: ntpd: command not found 2026-02-13T10:35:39.554 INFO:teuthology.orchestra.run.trial121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T10:35:39.554 INFO:teuthology.orchestra.run.trial163.stdout:506 Cannot talk to daemon 2026-02-13T10:35:39.566 INFO:teuthology.orchestra.run.trial163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T10:35:39.578 INFO:teuthology.orchestra.run.trial163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T10:35:39.599 INFO:teuthology.orchestra.run.trial121.stderr:bash: line 1: ntpq: command not found 2026-02-13T10:35:39.601 INFO:teuthology.orchestra.run.trial121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T10:35:39.601 INFO:teuthology.orchestra.run.trial121.stdout:=============================================================================== 2026-02-13T10:35:39.630 INFO:teuthology.orchestra.run.trial163.stderr:bash: line 1: ntpq: command not found 2026-02-13T10:35:39.647 INFO:teuthology.orchestra.run.trial163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T10:35:39.648 INFO:teuthology.orchestra.run.trial163.stdout:=============================================================================== 2026-02-13T10:35:39.648 INFO:teuthology.orchestra.run.trial163.stdout:^? h96-60-247-29.mdsnwi.bro> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T10:35:39.648 INFO:teuthology.orchestra.run.trial163.stdout:^? 149.28.61.105.vultruserc> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T10:35:39.648 INFO:teuthology.orchestra.run.trial163.stdout:^? blotch.image1tech.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T10:35:39.648 INFO:teuthology.orchestra.run.trial163.stdout:^? frumentum-zeus.infra.fru> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T10:35:39.649 INFO:teuthology.run_tasks:Running task install... 2026-02-13T10:35:39.662 DEBUG:teuthology.task.install:project ceph 2026-02-13T10:35:39.663 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '79a8b77b79a4b70f9c11cf81d24d449d40916c58'}} 2026-02-13T10:35:39.663 DEBUG:teuthology.task.install:config {'branch': 'squid', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '79a8b77b79a4b70f9c11cf81d24d449d40916c58'} 2026-02-13T10:35:39.663 INFO:teuthology.task.install:Using flavor: default 2026-02-13T10:35:39.671 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-13T10:35:39.671 INFO:teuthology.task.install:extra packages: [] 2026-02-13T10:35:39.672 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'squid', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79a8b77b79a4b70f9c11cf81d24d449d40916c58', 'tag': None, 'wait_for_package': False} 2026-02-13T10:35:39.672 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T10:35:39.673 INFO:teuthology.packaging:ref: None 2026-02-13T10:35:39.673 INFO:teuthology.packaging:tag: None 2026-02-13T10:35:39.673 INFO:teuthology.packaging:branch: squid 2026-02-13T10:35:39.673 INFO:teuthology.packaging:sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:35:39.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2026-02-13T10:35:39.675 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'squid', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79a8b77b79a4b70f9c11cf81d24d449d40916c58', 'tag': None, 'wait_for_package': False} 2026-02-13T10:35:39.676 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T10:35:39.676 INFO:teuthology.packaging:ref: None 2026-02-13T10:35:39.676 INFO:teuthology.packaging:tag: None 2026-02-13T10:35:39.676 INFO:teuthology.packaging:branch: squid 2026-02-13T10:35:39.676 INFO:teuthology.packaging:sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:35:39.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2026-02-13T10:35:39.906 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/ 2026-02-13T10:35:39.907 INFO:teuthology.task.install.rpm:Package version is 19.2.3-672.g283431c0 2026-02-13T10:35:39.938 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/ 2026-02-13T10:35:39.938 INFO:teuthology.task.install.rpm:Package version is 19.2.3-672.g283431c0 2026-02-13T10:35:40.115 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T10:35:40.115 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:35:40.115 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T10:35:40.129 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T10:35:40.129 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:35:40.129 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T10:35:40.140 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-13T10:35:40.140 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T10:35:40.140 INFO:teuthology.packaging:ref: None 2026-02-13T10:35:40.140 INFO:teuthology.packaging:tag: None 2026-02-13T10:35:40.140 INFO:teuthology.packaging:branch: squid 2026-02-13T10:35:40.141 INFO:teuthology.packaging:sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:35:40.141 DEBUG:teuthology.orchestra.run.trial121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/squid/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T10:35:40.154 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-13T10:35:40.154 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T10:35:40.155 INFO:teuthology.packaging:ref: None 2026-02-13T10:35:40.155 INFO:teuthology.packaging:tag: None 2026-02-13T10:35:40.155 INFO:teuthology.packaging:branch: squid 2026-02-13T10:35:40.155 INFO:teuthology.packaging:sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:35:40.155 DEBUG:teuthology.orchestra.run.trial163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/squid/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T10:35:40.204 DEBUG:teuthology.orchestra.run.trial121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T10:35:40.220 DEBUG:teuthology.orchestra.run.trial163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T10:35:40.277 DEBUG:teuthology.orchestra.run.trial121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T10:35:40.294 DEBUG:teuthology.orchestra.run.trial163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T10:35:40.340 INFO:teuthology.orchestra.run.trial121.stdout:check_obsoletes = 1 2026-02-13T10:35:40.341 DEBUG:teuthology.orchestra.run.trial121:> sudo yum clean all 2026-02-13T10:35:40.356 INFO:teuthology.orchestra.run.trial163.stdout:check_obsoletes = 1 2026-02-13T10:35:40.358 DEBUG:teuthology.orchestra.run.trial163:> sudo yum clean all 2026-02-13T10:35:40.491 INFO:teuthology.orchestra.run.trial121.stdout:41 files removed 2026-02-13T10:35:40.506 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-radosgw 2026-02-13T10:35:40.509 INFO:teuthology.orchestra.run.trial163.stdout:41 files removed 2026-02-13T10:35:40.524 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-radosgw 2026-02-13T10:35:40.776 INFO:teuthology.orchestra.run.trial121.stdout:ceph packages for x86_64 652 kB/s | 84 kB 00:00 2026-02-13T10:35:40.796 INFO:teuthology.orchestra.run.trial163.stdout:ceph packages for x86_64 653 kB/s | 84 kB 00:00 2026-02-13T10:35:40.907 INFO:teuthology.orchestra.run.trial121.stdout:ceph noarch packages 111 kB/s | 12 kB 00:00 2026-02-13T10:35:40.928 INFO:teuthology.orchestra.run.trial163.stdout:ceph noarch packages 109 kB/s | 12 kB 00:00 2026-02-13T10:35:41.032 INFO:teuthology.orchestra.run.trial121.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-13T10:35:41.053 INFO:teuthology.orchestra.run.trial163.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-13T10:35:41.754 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-13T10:35:41.879 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-13T10:35:42.675 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - AppStream 75 MB/s | 27 MB 00:00 2026-02-13T10:35:44.903 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-13T10:35:47.427 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 7.8 MB 00:02 2026-02-13T10:35:49.170 INFO:teuthology.orchestra.run.trial163.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:01 2026-02-13T10:35:52.385 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 7.8 MB 00:05 2026-02-13T10:35:53.348 INFO:teuthology.orchestra.run.trial121.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 20 kB 00:00 2026-02-13T10:35:53.901 INFO:teuthology.orchestra.run.trial121.stdout:Extra Packages for Enterprise Linux 42 MB/s | 20 MB 00:00 2026-02-13T10:35:56.786 INFO:teuthology.orchestra.run.trial163.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 20 MB 00:07 2026-02-13T10:35:57.221 INFO:teuthology.orchestra.run.trial121.stdout:lab-extras 1.0 MB/s | 50 kB 00:00 2026-02-13T10:35:58.507 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout: ceph-radosgw x86_64 2:19.2.3-672.g283431c0.el9 ceph 11 M 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout:Upgrading: 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout: librados2 x86_64 2:19.2.3-672.g283431c0.el9 ceph 3.4 M 2026-02-13T10:35:58.509 INFO:teuthology.orchestra.run.trial121.stdout: librbd1 x86_64 2:19.2.3-672.g283431c0.el9 ceph 3.2 M 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: ceph-base x86_64 2:19.2.3-672.g283431c0.el9 ceph 5.5 M 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: ceph-common x86_64 2:19.2.3-672.g283431c0.el9 ceph 22 M 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: ceph-selinux x86_64 2:19.2.3-672.g283431c0.el9 ceph 25 k 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: libcephfs2 x86_64 2:19.2.3-672.g283431c0.el9 ceph 1.0 M 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T10:35:58.510 INFO:teuthology.orchestra.run.trial121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: libradosstriper1 x86_64 2:19.2.3-672.g283431c0.el9 ceph 504 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: librgw2 x86_64 2:19.2.3-672.g283431c0.el9 ceph 5.4 M 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: python3-ceph-argparse x86_64 2:19.2.3-672.g283431c0.el9 ceph 45 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: python3-ceph-common x86_64 2:19.2.3-672.g283431c0.el9 ceph 142 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: python3-cephfs x86_64 2:19.2.3-672.g283431c0.el9 ceph 165 k 2026-02-13T10:35:58.511 INFO:teuthology.orchestra.run.trial121.stdout: python3-rados x86_64 2:19.2.3-672.g283431c0.el9 ceph 323 k 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: python3-rbd x86_64 2:19.2.3-672.g283431c0.el9 ceph 302 k 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: python3-rgw x86_64 2:19.2.3-672.g283431c0.el9 ceph 100 k 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout:Installing weak dependencies: 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout:Install 32 Packages 2026-02-13T10:35:58.512 INFO:teuthology.orchestra.run.trial121.stdout:Upgrade 2 Packages 2026-02-13T10:35:58.513 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:35:58.513 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 62 M 2026-02-13T10:35:58.513 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:35:59.144 INFO:teuthology.orchestra.run.trial121.stdout:(1/34): ceph-base-19.2.3-672.g283431c0.el9.x86_ 13 MB/s | 5.5 MB 00:00 2026-02-13T10:35:59.184 INFO:teuthology.orchestra.run.trial121.stdout:(2/34): ceph-selinux-19.2.3-672.g283431c0.el9.x 641 kB/s | 25 kB 00:00 2026-02-13T10:35:59.284 INFO:teuthology.orchestra.run.trial121.stdout:(3/34): libcephfs2-19.2.3-672.g283431c0.el9.x86 9.8 MB/s | 1.0 MB 00:00 2026-02-13T10:35:59.322 INFO:teuthology.orchestra.run.trial121.stdout:(4/34): ceph-radosgw-19.2.3-672.g283431c0.el9.x 18 MB/s | 11 MB 00:00 2026-02-13T10:35:59.331 INFO:teuthology.orchestra.run.trial121.stdout:(5/34): libradosstriper1-19.2.3-672.g283431c0.e 10 MB/s | 504 kB 00:00 2026-02-13T10:35:59.377 INFO:teuthology.orchestra.run.trial121.stdout:(6/34): python3-ceph-argparse-19.2.3-672.g28343 978 kB/s | 45 kB 00:00 2026-02-13T10:35:59.429 INFO:teuthology.orchestra.run.trial121.stdout:(7/34): python3-ceph-common-19.2.3-672.g283431c 2.7 MB/s | 142 kB 00:00 2026-02-13T10:35:59.483 INFO:teuthology.orchestra.run.trial121.stdout:(8/34): python3-cephfs-19.2.3-672.g283431c0.el9 2.9 MB/s | 165 kB 00:00 2026-02-13T10:35:59.534 INFO:teuthology.orchestra.run.trial121.stdout:(9/34): librgw2-19.2.3-672.g283431c0.el9.x86_64 26 MB/s | 5.4 MB 00:00 2026-02-13T10:35:59.541 INFO:teuthology.orchestra.run.trial121.stdout:(10/34): python3-rados-19.2.3-672.g283431c0.el9 5.5 MB/s | 323 kB 00:00 2026-02-13T10:35:59.574 INFO:teuthology.orchestra.run.trial121.stdout:(11/34): python3-rbd-19.2.3-672.g283431c0.el9.x 7.4 MB/s | 302 kB 00:00 2026-02-13T10:35:59.577 INFO:teuthology.orchestra.run.trial121.stdout:(12/34): python3-rgw-19.2.3-672.g283431c0.el9.x 2.7 MB/s | 100 kB 00:00 2026-02-13T10:35:59.640 INFO:teuthology.orchestra.run.trial121.stdout:(13/34): ceph-common-19.2.3-672.g283431c0.el9.x 24 MB/s | 22 MB 00:00 2026-02-13T10:35:59.641 INFO:teuthology.orchestra.run.trial121.stdout:(14/34): mailcap-2.1.49-5.el9.noarch.rpm 499 kB/s | 33 kB 00:00 2026-02-13T10:35:59.648 INFO:teuthology.orchestra.run.trial121.stdout:(15/34): pciutils-3.7.0-7.el9.x86_64.rpm 1.3 MB/s | 93 kB 00:00 2026-02-13T10:36:00.023 INFO:teuthology.orchestra.run.trial163.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-13T10:36:00.442 INFO:teuthology.orchestra.run.trial121.stdout:(16/34): boost-program-options-1.75.0-13.el9.x8 130 kB/s | 104 kB 00:00 2026-02-13T10:36:00.448 INFO:teuthology.orchestra.run.trial121.stdout:(17/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2026-02-13T10:36:00.600 INFO:teuthology.orchestra.run.trial121.stdout:(18/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 286 kB/s | 45 kB 00:00 2026-02-13T10:36:00.763 INFO:teuthology.orchestra.run.trial121.stdout:(19/34): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2026-02-13T10:36:00.836 INFO:teuthology.orchestra.run.trial121.stdout:(20/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2026-02-13T10:36:00.841 INFO:teuthology.orchestra.run.trial121.stdout:(21/34): libnbd-1.20.3-4.el9.x86_64.rpm 137 kB/s | 164 kB 00:01 2026-02-13T10:36:00.993 INFO:teuthology.orchestra.run.trial121.stdout:(22/34): qatlib-service-25.08.0-2.el9.x86_64.rp 236 kB/s | 37 kB 00:00 2026-02-13T10:36:00.998 INFO:teuthology.orchestra.run.trial121.stdout:(23/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 421 kB/s | 66 kB 00:00 2026-02-13T10:36:01.224 INFO:teuthology.orchestra.run.trial121.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.3 MB/s | 308 kB 00:00 2026-02-13T10:36:01.313 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout: ceph-radosgw x86_64 2:19.2.3-672.g283431c0.el9 ceph 11 M 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout:Upgrading: 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout: librados2 x86_64 2:19.2.3-672.g283431c0.el9 ceph 3.4 M 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout: librbd1 x86_64 2:19.2.3-672.g283431c0.el9 ceph 3.2 M 2026-02-13T10:36:01.315 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: ceph-base x86_64 2:19.2.3-672.g283431c0.el9 ceph 5.5 M 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: ceph-common x86_64 2:19.2.3-672.g283431c0.el9 ceph 22 M 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: ceph-selinux x86_64 2:19.2.3-672.g283431c0.el9 ceph 25 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: libcephfs2 x86_64 2:19.2.3-672.g283431c0.el9 ceph 1.0 M 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T10:36:01.316 INFO:teuthology.orchestra.run.trial163.stdout: libradosstriper1 x86_64 2:19.2.3-672.g283431c0.el9 ceph 504 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: librgw2 x86_64 2:19.2.3-672.g283431c0.el9 ceph 5.4 M 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: python3-ceph-argparse x86_64 2:19.2.3-672.g283431c0.el9 ceph 45 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: python3-ceph-common x86_64 2:19.2.3-672.g283431c0.el9 ceph 142 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: python3-cephfs x86_64 2:19.2.3-672.g283431c0.el9 ceph 165 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: python3-rados x86_64 2:19.2.3-672.g283431c0.el9 ceph 323 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: python3-rbd x86_64 2:19.2.3-672.g283431c0.el9 ceph 302 k 2026-02-13T10:36:01.317 INFO:teuthology.orchestra.run.trial163.stdout: python3-rgw x86_64 2:19.2.3-672.g283431c0.el9 ceph 100 k 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout:Installing weak dependencies: 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout:Install 32 Packages 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout:Upgrade 2 Packages 2026-02-13T10:36:01.318 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:01.319 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 62 M 2026-02-13T10:36:01.319 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:01.379 INFO:teuthology.orchestra.run.trial121.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 160 kB/s | 25 kB 00:00 2026-02-13T10:36:01.440 INFO:teuthology.orchestra.run.trial121.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 800 kB/s | 49 kB 00:00 2026-02-13T10:36:01.462 INFO:teuthology.orchestra.run.trial121.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 3.1 MB/s | 67 kB 00:00 2026-02-13T10:36:01.489 INFO:teuthology.orchestra.run.trial121.stdout:(28/34): libarrow-9.0.0-15.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2026-02-13T10:36:01.589 INFO:teuthology.orchestra.run.trial121.stdout:(29/34): qatlib-25.08.0-2.el9.x86_64.rpm 290 kB/s | 240 kB 00:00 2026-02-13T10:36:01.886 INFO:teuthology.orchestra.run.trial121.stdout:(30/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 1.9 MB/s | 838 kB 00:00 2026-02-13T10:36:01.962 INFO:teuthology.orchestra.run.trial121.stdout:(31/34): re2-20211101-20.el9.x86_64.rpm 405 kB/s | 191 kB 00:00 2026-02-13T10:36:02.073 INFO:teuthology.orchestra.run.trial121.stdout:(32/34): librados2-19.2.3-672.g283431c0.el9.x86 18 MB/s | 3.4 MB 00:00 2026-02-13T10:36:02.197 INFO:teuthology.orchestra.run.trial163.stdout:(1/34): ceph-base-19.2.3-672.g283431c0.el9.x86_ 16 MB/s | 5.5 MB 00:00 2026-02-13T10:36:02.235 INFO:teuthology.orchestra.run.trial163.stdout:(2/34): ceph-selinux-19.2.3-672.g283431c0.el9.x 663 kB/s | 25 kB 00:00 2026-02-13T10:36:02.267 INFO:teuthology.orchestra.run.trial121.stdout:(33/34): librbd1-19.2.3-672.g283431c0.el9.x86_6 10 MB/s | 3.2 MB 00:00 2026-02-13T10:36:02.298 INFO:teuthology.orchestra.run.trial163.stdout:(3/34): libcephfs2-19.2.3-672.g283431c0.el9.x86 15 MB/s | 1.0 MB 00:00 2026-02-13T10:36:02.327 INFO:teuthology.orchestra.run.trial163.stdout:(4/34): libradosstriper1-19.2.3-672.g283431c0.e 17 MB/s | 504 kB 00:00 2026-02-13T10:36:02.533 INFO:teuthology.orchestra.run.trial163.stdout:(5/34): librgw2-19.2.3-672.g283431c0.el9.x86_64 26 MB/s | 5.4 MB 00:00 2026-02-13T10:36:02.559 INFO:teuthology.orchestra.run.trial163.stdout:(6/34): python3-ceph-argparse-19.2.3-672.g28343 1.7 MB/s | 45 kB 00:00 2026-02-13T10:36:02.588 INFO:teuthology.orchestra.run.trial163.stdout:(7/34): python3-ceph-common-19.2.3-672.g283431c 4.8 MB/s | 142 kB 00:00 2026-02-13T10:36:02.619 INFO:teuthology.orchestra.run.trial163.stdout:(8/34): python3-cephfs-19.2.3-672.g283431c0.el9 5.3 MB/s | 165 kB 00:00 2026-02-13T10:36:02.623 INFO:teuthology.orchestra.run.trial121.stdout:(34/34): thrift-0.15.0-4.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2026-02-13T10:36:02.631 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:02.631 INFO:teuthology.orchestra.run.trial121.stdout:Total 15 MB/s | 62 MB 00:04 2026-02-13T10:36:02.654 INFO:teuthology.orchestra.run.trial163.stdout:(9/34): python3-rados-19.2.3-672.g283431c0.el9. 9.0 MB/s | 323 kB 00:00 2026-02-13T10:36:02.672 INFO:teuthology.orchestra.run.trial163.stdout:(10/34): ceph-radosgw-19.2.3-672.g283431c0.el9. 13 MB/s | 11 MB 00:00 2026-02-13T10:36:02.679 INFO:teuthology.orchestra.run.trial163.stdout:(11/34): python3-rbd-19.2.3-672.g283431c0.el9.x 12 MB/s | 302 kB 00:00 2026-02-13T10:36:02.696 INFO:teuthology.orchestra.run.trial163.stdout:(12/34): python3-rgw-19.2.3-672.g283431c0.el9.x 4.1 MB/s | 100 kB 00:00 2026-02-13T10:36:02.704 INFO:teuthology.orchestra.run.trial163.stdout:(13/34): mailcap-2.1.49-5.el9.noarch.rpm 1.3 MB/s | 33 kB 00:00 2026-02-13T10:36:02.722 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:02.733 INFO:teuthology.orchestra.run.trial163.stdout:(14/34): pciutils-3.7.0-7.el9.x86_64.rpm 2.5 MB/s | 93 kB 00:00 2026-02-13T10:36:02.739 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:02.739 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:02.862 INFO:teuthology.orchestra.run.trial163.stdout:(15/34): ceph-common-19.2.3-672.g283431c0.el9.x 22 MB/s | 22 MB 00:00 2026-02-13T10:36:02.863 INFO:teuthology.orchestra.run.trial163.stdout:(16/34): boost-program-options-1.75.0-13.el9.x8 659 kB/s | 104 kB 00:00 2026-02-13T10:36:02.873 INFO:teuthology.orchestra.run.trial163.stdout:(17/34): libnbd-1.20.3-4.el9.x86_64.rpm 1.1 MB/s | 164 kB 00:00 2026-02-13T10:36:02.917 INFO:teuthology.orchestra.run.trial163.stdout:(18/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.9 MB/s | 160 kB 00:00 2026-02-13T10:36:02.957 INFO:teuthology.orchestra.run.trial163.stdout:(19/34): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2026-02-13T10:36:02.960 INFO:teuthology.orchestra.run.trial163.stdout:(20/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 464 kB/s | 45 kB 00:00 2026-02-13T10:36:02.968 INFO:teuthology.orchestra.run.trial163.stdout:(21/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 292 kB 00:00 2026-02-13T10:36:02.983 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:02.983 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:02.986 INFO:teuthology.orchestra.run.trial163.stdout:(22/34): qatlib-25.08.0-2.el9.x86_64.rpm 8.0 MB/s | 240 kB 00:00 2026-02-13T10:36:02.994 INFO:teuthology.orchestra.run.trial163.stdout:(23/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.5 MB/s | 66 kB 00:00 2026-02-13T10:36:02.995 INFO:teuthology.orchestra.run.trial163.stdout:(24/34): qatlib-service-25.08.0-2.el9.x86_64.rp 1.1 MB/s | 37 kB 00:00 2026-02-13T10:36:03.183 INFO:teuthology.orchestra.run.trial163.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 131 kB/s | 25 kB 00:00 2026-02-13T10:36:03.268 INFO:teuthology.orchestra.run.trial163.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2026-02-13T10:36:03.268 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:03.277 INFO:teuthology.orchestra.run.trial121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-13T10:36:03.279 INFO:teuthology.orchestra.run.trial121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/36 2026-02-13T10:36:03.291 INFO:teuthology.orchestra.run.trial121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/36 2026-02-13T10:36:03.325 INFO:teuthology.orchestra.run.trial163.stdout:(27/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 909 kB/s | 308 kB 00:00 2026-02-13T10:36:03.330 INFO:teuthology.orchestra.run.trial163.stdout:(28/34): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2026-02-13T10:36:03.432 INFO:teuthology.orchestra.run.trial163.stdout:(29/34): re2-20211101-20.el9.x86_64.rpm 1.8 MB/s | 191 kB 00:00 2026-02-13T10:36:03.448 INFO:teuthology.orchestra.run.trial121.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/36 2026-02-13T10:36:03.452 INFO:teuthology.orchestra.run.trial121.stdout: Upgrading : librados2-2:19.2.3-672.g283431c0.el9.x86_64 5/36 2026-02-13T10:36:03.452 INFO:teuthology.orchestra.run.trial163.stdout:(30/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 6.4 MB/s | 838 kB 00:00 2026-02-13T10:36:03.481 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: librados2-2:19.2.3-672.g283431c0.el9.x86_64 5/36 2026-02-13T10:36:03.483 INFO:teuthology.orchestra.run.trial163.stdout:(31/34): libarrow-9.0.0-15.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2026-02-13T10:36:03.526 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 6/36 2026-02-13T10:36:03.527 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 7/36 2026-02-13T10:36:03.539 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 7/36 2026-02-13T10:36:03.543 INFO:teuthology.orchestra.run.trial121.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/36 2026-02-13T10:36:03.551 INFO:teuthology.orchestra.run.trial163.stdout:(32/34): librados2-19.2.3-672.g283431c0.el9.x86 35 MB/s | 3.4 MB 00:00 2026-02-13T10:36:03.553 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-ceph-argparse-2:19.2.3-672.g283431c0.el9.x 9/36 2026-02-13T10:36:03.576 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 10/36 2026-02-13T10:36:03.577 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 11/36 2026-02-13T10:36:03.584 INFO:teuthology.orchestra.run.trial163.stdout:(33/34): librbd1-19.2.3-672.g283431c0.el9.x86_6 32 MB/s | 3.2 MB 00:00 2026-02-13T10:36:03.590 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 11/36 2026-02-13T10:36:03.605 INFO:teuthology.orchestra.run.trial163.stdout:(34/34): thrift-0.15.0-4.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2026-02-13T10:36:03.609 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:03.609 INFO:teuthology.orchestra.run.trial163.stdout:Total 27 MB/s | 62 MB 00:02 2026-02-13T10:36:03.624 INFO:teuthology.orchestra.run.trial121.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/36 2026-02-13T10:36:03.628 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/36 2026-02-13T10:36:03.634 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/36 2026-02-13T10:36:03.636 INFO:teuthology.orchestra.run.trial121.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/36 2026-02-13T10:36:03.639 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/36 2026-02-13T10:36:03.644 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/36 2026-02-13T10:36:03.698 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:03.715 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:03.715 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:03.791 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-13T10:36:03.792 INFO:teuthology.orchestra.run.trial121.stdout: Upgrading : librbd1-2:19.2.3-672.g283431c0.el9.x86_64 19/36 2026-02-13T10:36:03.821 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: librbd1-2:19.2.3-672.g283431c0.el9.x86_64 19/36 2026-02-13T10:36:03.823 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 20/36 2026-02-13T10:36:03.830 INFO:teuthology.orchestra.run.trial121.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-13T10:36:03.959 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:03.959 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:04.063 INFO:teuthology.orchestra.run.trial121.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-13T10:36:04.064 INFO:teuthology.orchestra.run.trial121.stdout: Installing : librgw2-2:19.2.3-672.g283431c0.el9.x86_64 23/36 2026-02-13T10:36:04.080 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: librgw2-2:19.2.3-672.g283431c0.el9.x86_64 23/36 2026-02-13T10:36:04.084 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 24/36 2026-02-13T10:36:04.088 INFO:teuthology.orchestra.run.trial121.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-13T10:36:04.090 INFO:teuthology.orchestra.run.trial121.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-13T10:36:04.091 INFO:teuthology.orchestra.run.trial121.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T10:36:04.109 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T10:36:04.112 INFO:teuthology.orchestra.run.trial121.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-13T10:36:04.139 INFO:teuthology.orchestra.run.trial121.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-13T10:36:04.141 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-ceph-common-2:19.2.3-672.g283431c0.el9.x86 30/36 2026-02-13T10:36:04.246 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:04.255 INFO:teuthology.orchestra.run.trial163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-13T10:36:04.258 INFO:teuthology.orchestra.run.trial163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/36 2026-02-13T10:36:04.269 INFO:teuthology.orchestra.run.trial163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/36 2026-02-13T10:36:04.419 INFO:teuthology.orchestra.run.trial163.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/36 2026-02-13T10:36:04.423 INFO:teuthology.orchestra.run.trial163.stdout: Upgrading : librados2-2:19.2.3-672.g283431c0.el9.x86_64 5/36 2026-02-13T10:36:04.454 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: librados2-2:19.2.3-672.g283431c0.el9.x86_64 5/36 2026-02-13T10:36:04.497 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 6/36 2026-02-13T10:36:04.498 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 7/36 2026-02-13T10:36:04.509 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 7/36 2026-02-13T10:36:04.514 INFO:teuthology.orchestra.run.trial163.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/36 2026-02-13T10:36:04.522 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-ceph-argparse-2:19.2.3-672.g283431c0.el9.x 9/36 2026-02-13T10:36:04.545 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 10/36 2026-02-13T10:36:04.546 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 11/36 2026-02-13T10:36:04.559 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 11/36 2026-02-13T10:36:04.596 INFO:teuthology.orchestra.run.trial163.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/36 2026-02-13T10:36:04.600 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/36 2026-02-13T10:36:04.606 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/36 2026-02-13T10:36:04.608 INFO:teuthology.orchestra.run.trial163.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/36 2026-02-13T10:36:04.611 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/36 2026-02-13T10:36:04.616 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/36 2026-02-13T10:36:04.756 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-13T10:36:04.757 INFO:teuthology.orchestra.run.trial163.stdout: Upgrading : librbd1-2:19.2.3-672.g283431c0.el9.x86_64 19/36 2026-02-13T10:36:04.783 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: librbd1-2:19.2.3-672.g283431c0.el9.x86_64 19/36 2026-02-13T10:36:04.786 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 20/36 2026-02-13T10:36:04.792 INFO:teuthology.orchestra.run.trial163.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-13T10:36:05.016 INFO:teuthology.orchestra.run.trial163.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-13T10:36:05.017 INFO:teuthology.orchestra.run.trial163.stdout: Installing : librgw2-2:19.2.3-672.g283431c0.el9.x86_64 23/36 2026-02-13T10:36:05.033 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: librgw2-2:19.2.3-672.g283431c0.el9.x86_64 23/36 2026-02-13T10:36:05.037 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 24/36 2026-02-13T10:36:05.041 INFO:teuthology.orchestra.run.trial163.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-13T10:36:05.044 INFO:teuthology.orchestra.run.trial163.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-13T10:36:05.044 INFO:teuthology.orchestra.run.trial163.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T10:36:05.063 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-13T10:36:05.065 INFO:teuthology.orchestra.run.trial163.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-13T10:36:05.093 INFO:teuthology.orchestra.run.trial163.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-13T10:36:05.095 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-ceph-common-2:19.2.3-672.g283431c0.el9.x86 30/36 2026-02-13T10:36:05.181 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 31/36 2026-02-13T10:36:05.184 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 31/36 2026-02-13T10:36:05.477 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 31/36 2026-02-13T10:36:05.481 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 32/36 2026-02-13T10:36:05.523 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 32/36 2026-02-13T10:36:05.523 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T10:36:05.523 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T10:36:05.523 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:05.526 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 33/36 2026-02-13T10:36:06.079 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 31/36 2026-02-13T10:36:06.082 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 31/36 2026-02-13T10:36:06.368 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 31/36 2026-02-13T10:36:06.371 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 32/36 2026-02-13T10:36:06.412 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 32/36 2026-02-13T10:36:06.412 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T10:36:06.412 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T10:36:06.412 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:06.415 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 33/36 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 33/36 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout:skipping the directory /sys 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout:skipping the directory /proc 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout:skipping the directory /mnt 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout:skipping the directory /var/tmp 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout:skipping the directory /home 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout:skipping the directory /root 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout:skipping the directory /tmp 2026-02-13T10:36:10.042 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:10.520 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 34/36 2026-02-13T10:36:10.543 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 34/36 2026-02-13T10:36:10.543 INFO:teuthology.orchestra.run.trial121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:10.543 INFO:teuthology.orchestra.run.trial121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T10:36:10.543 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T10:36:10.543 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T10:36:10.543 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:10.545 INFO:teuthology.orchestra.run.trial121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T10:36:10.557 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T10:36:10.557 INFO:teuthology.orchestra.run.trial121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T10:36:10.613 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T10:36:10.613 INFO:teuthology.orchestra.run.trial121.stdout:Creating group 'qat' with GID 992. 2026-02-13T10:36:10.613 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:10.885 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 33/36 2026-02-13T10:36:10.885 INFO:teuthology.orchestra.run.trial163.stdout:skipping the directory /sys 2026-02-13T10:36:10.885 INFO:teuthology.orchestra.run.trial163.stdout:skipping the directory /proc 2026-02-13T10:36:10.885 INFO:teuthology.orchestra.run.trial163.stdout:skipping the directory /mnt 2026-02-13T10:36:10.885 INFO:teuthology.orchestra.run.trial163.stdout:skipping the directory /var/tmp 2026-02-13T10:36:10.886 INFO:teuthology.orchestra.run.trial163.stdout:skipping the directory /home 2026-02-13T10:36:10.886 INFO:teuthology.orchestra.run.trial163.stdout:skipping the directory /root 2026-02-13T10:36:10.886 INFO:teuthology.orchestra.run.trial163.stdout:skipping the directory /tmp 2026-02-13T10:36:10.886 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:10.889 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 1/36 2026-02-13T10:36:10.889 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 2/36 2026-02-13T10:36:10.889 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 3/36 2026-02-13T10:36:10.889 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 4/36 2026-02-13T10:36:10.889 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 5/36 2026-02-13T10:36:10.889 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 6/36 2026-02-13T10:36:10.889 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librgw2-2:19.2.3-672.g283431c0.el9.x86_64 7/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-ceph-argparse-2:19.2.3-672.g283431c0.el9.x 8/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-ceph-common-2:19.2.3-672.g283431c0.el9.x86 9/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 10/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 11/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 12/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 13/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-13T10:36:10.890 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-13T10:36:10.891 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-13T10:36:10.892 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librados2-2:19.2.3-672.g283431c0.el9.x86_64 33/36 2026-02-13T10:36:10.892 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-13T10:36:10.892 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librbd1-2:19.2.3-672.g283431c0.el9.x86_64 35/36 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout:Upgraded: 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: librados2-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: librbd1-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.947 INFO:teuthology.orchestra.run.trial121.stdout: ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: librgw2-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.948 INFO:teuthology.orchestra.run.trial121.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: python3-ceph-argparse-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: python3-ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T10:36:10.949 INFO:teuthology.orchestra.run.trial121.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T10:36:10.950 INFO:teuthology.orchestra.run.trial121.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T10:36:10.950 INFO:teuthology.orchestra.run.trial121.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T10:36:10.950 INFO:teuthology.orchestra.run.trial121.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T10:36:10.950 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:10.950 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:10.988 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-test 2026-02-13T10:36:11.285 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:14 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:11.337 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 34/36 2026-02-13T10:36:11.360 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 34/36 2026-02-13T10:36:11.360 INFO:teuthology.orchestra.run.trial163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:11.361 INFO:teuthology.orchestra.run.trial163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T10:36:11.361 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T10:36:11.361 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T10:36:11.361 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:11.363 INFO:teuthology.orchestra.run.trial163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T10:36:11.369 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:11.369 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:11.369 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:36:11.369 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:11.369 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:11.369 INFO:teuthology.orchestra.run.trial121.stdout: ceph-test x86_64 2:19.2.3-672.g283431c0.el9 ceph 50 M 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout: libcephsqlite x86_64 2:19.2.3-672.g283431c0.el9 ceph 163 k 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout:Install 5 Packages 2026-02-13T10:36:11.370 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:11.371 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 50 M 2026-02-13T10:36:11.371 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 213 M 2026-02-13T10:36:11.371 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:11.375 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-13T10:36:11.375 INFO:teuthology.orchestra.run.trial163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T10:36:11.430 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T10:36:11.430 INFO:teuthology.orchestra.run.trial163.stdout:Creating group 'qat' with GID 992. 2026-02-13T10:36:11.430 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:11.710 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 1/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 2/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 3/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 4/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 5/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 6/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librgw2-2:19.2.3-672.g283431c0.el9.x86_64 7/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-ceph-argparse-2:19.2.3-672.g283431c0.el9.x 8/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-ceph-common-2:19.2.3-672.g283431c0.el9.x86 9/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 10/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 11/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 12/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 13/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librados2-2:19.2.3-672.g283431c0.el9.x86_64 33/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-13T10:36:11.711 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librbd1-2:19.2.3-672.g283431c0.el9.x86_64 35/36 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout:Upgraded: 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: librados2-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: librbd1-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: ceph-radosgw-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: ceph-selinux-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T10:36:11.771 INFO:teuthology.orchestra.run.trial163.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: libradosstriper1-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: librgw2-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: python3-ceph-argparse-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: python3-ceph-common-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:11.772 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:11.814 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-test 2026-02-13T10:36:11.839 INFO:teuthology.orchestra.run.trial121.stdout:(1/5): libcephsqlite-19.2.3-672.g283431c0.el9.x 1.7 MB/s | 163 kB 00:00 2026-02-13T10:36:12.118 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:12 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:12.200 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:12.200 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout: ceph-test x86_64 2:19.2.3-672.g283431c0.el9 ceph 50 M 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout: libcephsqlite x86_64 2:19.2.3-672.g283431c0.el9 ceph 163 k 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:12.201 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:12.202 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:12.202 INFO:teuthology.orchestra.run.trial163.stdout:Install 5 Packages 2026-02-13T10:36:12.202 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:12.202 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 50 M 2026-02-13T10:36:12.202 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 213 M 2026-02-13T10:36:12.202 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:12.477 INFO:teuthology.orchestra.run.trial121.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 317 kB/s | 233 kB 00:00 2026-02-13T10:36:12.495 INFO:teuthology.orchestra.run.trial121.stdout:(3/5): socat-1.7.4.1-8.el9.x86_64.rpm 462 kB/s | 303 kB 00:00 2026-02-13T10:36:12.563 INFO:teuthology.orchestra.run.trial163.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 4.0 MB/s | 233 kB 00:00 2026-02-13T10:36:12.577 INFO:teuthology.orchestra.run.trial121.stdout:(4/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 639 kB/s | 64 kB 00:00 2026-02-13T10:36:12.581 INFO:teuthology.orchestra.run.trial163.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 16 MB/s | 303 kB 00:00 2026-02-13T10:36:12.590 INFO:teuthology.orchestra.run.trial163.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 6.9 MB/s | 64 kB 00:00 2026-02-13T10:36:12.728 INFO:teuthology.orchestra.run.trial163.stdout:(4/5): libcephsqlite-19.2.3-672.g283431c0.el9.x 735 kB/s | 163 kB 00:00 2026-02-13T10:36:12.761 INFO:teuthology.orchestra.run.trial121.stdout:(5/5): ceph-test-19.2.3-672.g283431c0.el9.x86_6 49 MB/s | 50 MB 00:01 2026-02-13T10:36:12.764 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:12.764 INFO:teuthology.orchestra.run.trial121.stdout:Total 36 MB/s | 50 MB 00:01 2026-02-13T10:36:12.788 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:12.797 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:12.797 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:12.982 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:12.982 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:13.192 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:13.200 INFO:teuthology.orchestra.run.trial121.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-13T10:36:13.205 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-13T10:36:13.214 INFO:teuthology.orchestra.run.trial121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-13T10:36:13.214 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 4/5 2026-02-13T10:36:13.654 INFO:teuthology.orchestra.run.trial163.stdout:(5/5): ceph-test-19.2.3-672.g283431c0.el9.x86_6 43 MB/s | 50 MB 00:01 2026-02-13T10:36:13.656 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:13.656 INFO:teuthology.orchestra.run.trial163.stdout:Total 35 MB/s | 50 MB 00:01 2026-02-13T10:36:13.683 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:13.691 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:13.692 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:13.876 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:13.876 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:14.087 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:14.095 INFO:teuthology.orchestra.run.trial163.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-13T10:36:14.100 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-13T10:36:14.107 INFO:teuthology.orchestra.run.trial163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-13T10:36:14.108 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 4/5 2026-02-13T10:36:15.500 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 4/5 2026-02-13T10:36:15.517 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 5/5 2026-02-13T10:36:15.764 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 5/5 2026-02-13T10:36:15.764 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 1/5 2026-02-13T10:36:15.764 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 2/5 2026-02-13T10:36:15.764 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-13T10:36:15.764 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-13T10:36:15.816 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-13T10:36:15.816 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout: ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout: libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:15.817 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:15.851 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph 2026-02-13T10:36:16.148 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:19 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:16.232 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout: ceph x86_64 2:19.2.3-672.g283431c0.el9 ceph 6.5 k 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mds x86_64 2:19.2.3-672.g283431c0.el9 ceph 2.4 M 2026-02-13T10:36:16.234 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr x86_64 2:19.2.3-672.g283431c0.el9 ceph 1.1 M 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-modules-core noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 253 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mon x86_64 2:19.2.3-672.g283431c0.el9 ceph 4.7 M 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: ceph-osd x86_64 2:19.2.3-672.g283431c0.el9 ceph 17 M 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T10:36:16.235 INFO:teuthology.orchestra.run.trial121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-13T10:36:16.236 INFO:teuthology.orchestra.run.trial121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T10:36:16.237 INFO:teuthology.orchestra.run.trial121.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout:Install 46 Packages 2026-02-13T10:36:16.238 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:16.239 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 31 M 2026-02-13T10:36:16.239 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 111 M 2026-02-13T10:36:16.239 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:16.356 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 4/5 2026-02-13T10:36:16.374 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 5/5 2026-02-13T10:36:16.599 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 5/5 2026-02-13T10:36:16.599 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 1/5 2026-02-13T10:36:16.599 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 2/5 2026-02-13T10:36:16.599 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-13T10:36:16.600 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-13T10:36:16.631 INFO:teuthology.orchestra.run.trial121.stdout:(1/46): ceph-19.2.3-672.g283431c0.el9.x86_64.rp 121 kB/s | 6.5 kB 00:00 2026-02-13T10:36:16.656 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-13T10:36:16.656 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout: ceph-test-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout: libcephsqlite-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:16.657 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:16.689 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph 2026-02-13T10:36:16.697 INFO:teuthology.orchestra.run.trial121.stdout:(2/46): ceph-mgr-19.2.3-672.g283431c0.el9.x86_6 9.0 MB/s | 1.1 MB 00:00 2026-02-13T10:36:16.752 INFO:teuthology.orchestra.run.trial121.stdout:(3/46): ceph-mds-19.2.3-672.g283431c0.el9.x86_6 14 MB/s | 2.4 MB 00:00 2026-02-13T10:36:16.788 INFO:teuthology.orchestra.run.trial121.stdout:(4/46): ceph-mgr-modules-core-19.2.3-672.g28343 6.9 MB/s | 253 kB 00:00 2026-02-13T10:36:16.961 INFO:teuthology.orchestra.run.trial121.stdout:(5/46): ceph-mon-19.2.3-672.g283431c0.el9.x86_6 14 MB/s | 4.7 MB 00:00 2026-02-13T10:36:16.990 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:16 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:17.074 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:17.075 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:17.075 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:17.075 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:17.075 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: ceph x86_64 2:19.2.3-672.g283431c0.el9 ceph 6.5 k 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mds x86_64 2:19.2.3-672.g283431c0.el9 ceph 2.4 M 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr x86_64 2:19.2.3-672.g283431c0.el9 ceph 1.1 M 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-modules-core noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 253 k 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mon x86_64 2:19.2.3-672.g283431c0.el9 ceph 4.7 M 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: ceph-osd x86_64 2:19.2.3-672.g283431c0.el9 ceph 17 M 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T10:36:17.076 INFO:teuthology.orchestra.run.trial163.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T10:36:17.077 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T10:36:17.078 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:17.079 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:17.080 INFO:teuthology.orchestra.run.trial163.stdout:Install 46 Packages 2026-02-13T10:36:17.080 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:17.080 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 31 M 2026-02-13T10:36:17.080 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 111 M 2026-02-13T10:36:17.081 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:17.119 INFO:teuthology.orchestra.run.trial121.stdout:(6/46): ceph-osd-19.2.3-672.g283431c0.el9.x86_6 41 MB/s | 17 MB 00:00 2026-02-13T10:36:17.389 INFO:teuthology.orchestra.run.trial121.stdout:(7/46): ledmon-libs-1.1.0-3.el9.x86_64.rpm 67 kB/s | 40 kB 00:00 2026-02-13T10:36:17.598 INFO:teuthology.orchestra.run.trial121.stdout:(8/46): libconfig-1.7.2-9.el9.x86_64.rpm 113 kB/s | 72 kB 00:00 2026-02-13T10:36:17.717 INFO:teuthology.orchestra.run.trial163.stdout:(1/46): ceph-19.2.3-672.g283431c0.el9.x86_64.rp 132 kB/s | 6.5 kB 00:00 2026-02-13T10:36:17.805 INFO:teuthology.orchestra.run.trial163.stdout:(2/46): ceph-mgr-19.2.3-672.g283431c0.el9.x86_6 7.8 MB/s | 1.1 MB 00:00 2026-02-13T10:36:17.837 INFO:teuthology.orchestra.run.trial163.stdout:(3/46): ceph-mds-19.2.3-672.g283431c0.el9.x86_6 14 MB/s | 2.4 MB 00:00 2026-02-13T10:36:17.887 INFO:teuthology.orchestra.run.trial163.stdout:(4/46): ceph-mgr-modules-core-19.2.3-672.g28343 4.9 MB/s | 253 kB 00:00 2026-02-13T10:36:17.895 INFO:teuthology.orchestra.run.trial121.stdout:(9/46): python3-cffi-1.14.5-5.el9.x86_64.rpm 326 kB/s | 253 kB 00:00 2026-02-13T10:36:17.923 INFO:teuthology.orchestra.run.trial163.stdout:(5/46): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.1 MB/s | 40 kB 00:00 2026-02-13T10:36:17.944 INFO:teuthology.orchestra.run.trial163.stdout:(6/46): ceph-mon-19.2.3-672.g283431c0.el9.x86_6 21 MB/s | 4.7 MB 00:00 2026-02-13T10:36:17.945 INFO:teuthology.orchestra.run.trial163.stdout:(7/46): libconfig-1.7.2-9.el9.x86_64.rpm 3.2 MB/s | 72 kB 00:00 2026-02-13T10:36:17.991 INFO:teuthology.orchestra.run.trial163.stdout:(8/46): python3-cryptography-36.0.1-5.el9.x86_6 27 MB/s | 1.2 MB 00:00 2026-02-13T10:36:17.997 INFO:teuthology.orchestra.run.trial163.stdout:(9/46): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 253 kB 00:00 2026-02-13T10:36:18.001 INFO:teuthology.orchestra.run.trial163.stdout:(10/46): python3-ply-3.11-14.el9.noarch.rpm 10 MB/s | 106 kB 00:00 2026-02-13T10:36:18.008 INFO:teuthology.orchestra.run.trial163.stdout:(11/46): python3-pycparser-2.20-6.el9.noarch.rp 12 MB/s | 135 kB 00:00 2026-02-13T10:36:18.011 INFO:teuthology.orchestra.run.trial163.stdout:(12/46): python3-pyparsing-2.4.7-9.el9.noarch.r 15 MB/s | 150 kB 00:00 2026-02-13T10:36:18.019 INFO:teuthology.orchestra.run.trial163.stdout:(13/46): python3-requests-2.25.1-10.el9.noarch. 11 MB/s | 126 kB 00:00 2026-02-13T10:36:18.023 INFO:teuthology.orchestra.run.trial163.stdout:(14/46): python3-urllib3-1.26.5-7.el9.noarch.rp 19 MB/s | 218 kB 00:00 2026-02-13T10:36:18.031 INFO:teuthology.orchestra.run.trial163.stdout:(15/46): unzip-6.0-59.el9.x86_64.rpm 15 MB/s | 182 kB 00:00 2026-02-13T10:36:18.034 INFO:teuthology.orchestra.run.trial163.stdout:(16/46): zip-3.0-35.el9.x86_64.rpm 24 MB/s | 266 kB 00:00 2026-02-13T10:36:18.206 INFO:teuthology.orchestra.run.trial163.stdout:(17/46): ceph-osd-19.2.3-672.g283431c0.el9.x86_ 43 MB/s | 17 MB 00:00 2026-02-13T10:36:18.269 INFO:teuthology.orchestra.run.trial121.stdout:(10/46): python3-ply-3.11-14.el9.noarch.rpm 159 kB/s | 106 kB 00:00 2026-02-13T10:36:18.341 INFO:teuthology.orchestra.run.trial121.stdout:(11/46): python3-cryptography-36.0.1-5.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2026-02-13T10:36:18.596 INFO:teuthology.orchestra.run.trial121.stdout:(12/46): python3-pycparser-2.20-6.el9.noarch.rp 193 kB/s | 135 kB 00:00 2026-02-13T10:36:18.767 INFO:teuthology.orchestra.run.trial163.stdout:(18/46): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2026-02-13T10:36:18.842 INFO:teuthology.orchestra.run.trial163.stdout:(19/46): python3-libstoragemgmt-1.10.1-1.el9.x8 278 kB/s | 177 kB 00:00 2026-02-13T10:36:18.842 INFO:teuthology.orchestra.run.trial163.stdout:(20/46): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 304 kB/s | 246 kB 00:00 2026-02-13T10:36:18.926 INFO:teuthology.orchestra.run.trial163.stdout:(21/46): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2026-02-13T10:36:18.978 INFO:teuthology.orchestra.run.trial121.stdout:(13/46): python3-pyparsing-2.4.7-9.el9.noarch.r 212 kB/s | 150 kB 00:00 2026-02-13T10:36:19.000 INFO:teuthology.orchestra.run.trial163.stdout:(22/46): python3-markupsafe-1.1.1-12.el9.x86_64 220 kB/s | 35 kB 00:00 2026-02-13T10:36:19.009 INFO:teuthology.orchestra.run.trial163.stdout:(23/46): lua-devel-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 22 kB 00:00 2026-02-13T10:36:19.033 INFO:teuthology.orchestra.run.trial121.stdout:(14/46): python3-requests-2.25.1-10.el9.noarch. 183 kB/s | 126 kB 00:00 2026-02-13T10:36:19.065 INFO:teuthology.orchestra.run.trial163.stdout:(24/46): luarocks-3.9.2-5.el9.noarch.rpm 2.7 MB/s | 151 kB 00:00 2026-02-13T10:36:19.080 INFO:teuthology.orchestra.run.trial163.stdout:(25/46): python3-packaging-20.9-5.el9.noarch.rp 325 kB/s | 77 kB 00:00 2026-02-13T10:36:19.088 INFO:teuthology.orchestra.run.trial163.stdout:(26/46): python3-toml-0.10.2-6.el9.noarch.rpm 256 kB/s | 42 kB 00:00 2026-02-13T10:36:19.107 INFO:teuthology.orchestra.run.trial163.stdout:(27/46): python3-autocommand-2.2.2-8.el9.noarch 696 kB/s | 29 kB 00:00 2026-02-13T10:36:19.108 INFO:teuthology.orchestra.run.trial163.stdout:(28/46): python3-backports-tarfile-1.2.0-1.el9. 2.1 MB/s | 60 kB 00:00 2026-02-13T10:36:19.126 INFO:teuthology.orchestra.run.trial163.stdout:(29/46): python3-cherrypy-18.6.1-2.el9.noarch.r 20 MB/s | 358 kB 00:00 2026-02-13T10:36:19.130 INFO:teuthology.orchestra.run.trial163.stdout:(30/46): python3-jaraco-8.2.1-3.el9.noarch.rpm 2.6 MB/s | 11 kB 00:00 2026-02-13T10:36:19.130 INFO:teuthology.orchestra.run.trial163.stdout:(31/46): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2026-02-13T10:36:19.132 INFO:teuthology.orchestra.run.trial163.stdout:(32/46): python3-cheroot-10.0.1-4.el9.noarch.rp 6.9 MB/s | 173 kB 00:00 2026-02-13T10:36:19.133 INFO:teuthology.orchestra.run.trial163.stdout:(33/46): python3-jaraco-classes-3.2.1-5.el9.noa 4.8 MB/s | 18 kB 00:00 2026-02-13T10:36:19.135 INFO:teuthology.orchestra.run.trial163.stdout:(34/46): python3-jaraco-collections-3.0.0-8.el9 5.3 MB/s | 23 kB 00:00 2026-02-13T10:36:19.138 INFO:teuthology.orchestra.run.trial163.stdout:(35/46): python3-jaraco-functools-3.5.0-2.el9.n 4.9 MB/s | 19 kB 00:00 2026-02-13T10:36:19.139 INFO:teuthology.orchestra.run.trial163.stdout:(36/46): python3-jaraco-text-4.0.0-2.el9.noarch 6.5 MB/s | 26 kB 00:00 2026-02-13T10:36:19.141 INFO:teuthology.orchestra.run.trial163.stdout:(37/46): python3-logutils-0.3.5-21.el9.noarch.r 12 MB/s | 46 kB 00:00 2026-02-13T10:36:19.143 INFO:teuthology.orchestra.run.trial163.stdout:(38/46): python3-jaraco-context-6.0.1-3.el9.noa 1.7 MB/s | 20 kB 00:00 2026-02-13T10:36:19.168 INFO:teuthology.orchestra.run.trial163.stdout:(39/46): python3-pecan-1.4.2-3.el9.noarch.rpm 10 MB/s | 272 kB 00:00 2026-02-13T10:36:19.169 INFO:teuthology.orchestra.run.trial163.stdout:(40/46): python3-portend-3.1.0-2.el9.noarch.rpm 641 kB/s | 16 kB 00:00 2026-02-13T10:36:19.172 INFO:teuthology.orchestra.run.trial163.stdout:(41/46): python3-more-itertools-8.12.0-2.el9.no 2.4 MB/s | 79 kB 00:00 2026-02-13T10:36:19.176 INFO:teuthology.orchestra.run.trial163.stdout:(42/46): python3-tempora-5.0.0-2.el9.noarch.rpm 4.9 MB/s | 36 kB 00:00 2026-02-13T10:36:19.186 INFO:teuthology.orchestra.run.trial163.stdout:(43/46): python3-webob-1.8.8-2.el9.noarch.rpm 16 MB/s | 230 kB 00:00 2026-02-13T10:36:19.191 INFO:teuthology.orchestra.run.trial163.stdout:(44/46): python3-zc-lockfile-2.0-10.el9.noarch. 4.5 MB/s | 20 kB 00:00 2026-02-13T10:36:19.195 INFO:teuthology.orchestra.run.trial163.stdout:(45/46): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.2 MB/s | 90 kB 00:00 2026-02-13T10:36:19.235 INFO:teuthology.orchestra.run.trial163.stdout:(46/46): python3-werkzeug-2.0.3-3.el9.1.noarch. 7.1 MB/s | 427 kB 00:00 2026-02-13T10:36:19.238 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:19.238 INFO:teuthology.orchestra.run.trial163.stdout:Total 15 MB/s | 31 MB 00:02 2026-02-13T10:36:19.343 INFO:teuthology.orchestra.run.trial121.stdout:(15/46): python3-urllib3-1.26.5-7.el9.noarch.rp 292 kB/s | 218 kB 00:00 2026-02-13T10:36:19.400 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:19.420 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:19.420 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:19.573 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:19.573 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:19.706 INFO:teuthology.orchestra.run.trial121.stdout:(16/46): unzip-6.0-59.el9.x86_64.rpm 250 kB/s | 182 kB 00:00 2026-02-13T10:36:19.763 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:19.765 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/46 2026-02-13T10:36:19.770 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/46 2026-02-13T10:36:19.775 INFO:teuthology.orchestra.run.trial163.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/46 2026-02-13T10:36:19.777 INFO:teuthology.orchestra.run.trial163.stdout: Installing : unzip-6.0-59.el9.x86_64 4/46 2026-02-13T10:36:19.783 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/46 2026-02-13T10:36:19.804 INFO:teuthology.orchestra.run.trial121.stdout:(17/46): zip-3.0-35.el9.x86_64.rpm 345 kB/s | 266 kB 00:00 2026-02-13T10:36:19.806 INFO:teuthology.orchestra.run.trial163.stdout: Installing : zip-3.0-35.el9.x86_64 6/46 2026-02-13T10:36:19.810 INFO:teuthology.orchestra.run.trial163.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/46 2026-02-13T10:36:19.815 INFO:teuthology.orchestra.run.trial163.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/46 2026-02-13T10:36:19.819 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/46 2026-02-13T10:36:19.850 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/46 2026-02-13T10:36:19.871 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 11/46 2026-02-13T10:36:19.880 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 12/46 2026-02-13T10:36:19.887 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 13/46 2026-02-13T10:36:19.891 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 14/46 2026-02-13T10:36:19.896 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 15/46 2026-02-13T10:36:19.902 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 16/46 2026-02-13T10:36:19.905 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/46 2026-02-13T10:36:19.911 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/46 2026-02-13T10:36:19.914 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 19/46 2026-02-13T10:36:19.921 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/46 2026-02-13T10:36:19.924 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 21/46 2026-02-13T10:36:19.930 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 22/46 2026-02-13T10:36:19.954 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/46 2026-02-13T10:36:20.008 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 24/46 2026-02-13T10:36:20.032 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 25/46 2026-02-13T10:36:20.045 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 26/46 2026-02-13T10:36:20.050 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 27/46 2026-02-13T10:36:20.060 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 28/46 2026-02-13T10:36:20.066 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 29/46 2026-02-13T10:36:20.079 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-ply-3.11-14.el9.noarch 30/46 2026-02-13T10:36:20.095 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 31/46 2026-02-13T10:36:20.130 INFO:teuthology.orchestra.run.trial121.stdout:(18/46): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 313 kB/s | 246 kB 00:00 2026-02-13T10:36:20.170 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 32/46 2026-02-13T10:36:20.178 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 33/46 2026-02-13T10:36:20.201 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 34/46 2026-02-13T10:36:20.231 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 35/46 2026-02-13T10:36:20.236 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 36/46 2026-02-13T10:36:20.282 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 37/46 2026-02-13T10:36:20.289 INFO:teuthology.orchestra.run.trial121.stdout:(19/46): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2026-02-13T10:36:20.332 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-672.g283431c0.el9.n 38/46 2026-02-13T10:36:20.333 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 39/46 2026-02-13T10:36:20.356 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 39/46 2026-02-13T10:36:20.356 INFO:teuthology.orchestra.run.trial163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:20.356 INFO:teuthology.orchestra.run.trial163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T10:36:20.356 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T10:36:20.356 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T10:36:20.356 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:20.360 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 40/46 2026-02-13T10:36:20.377 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 41/46 2026-02-13T10:36:20.377 INFO:teuthology.orchestra.run.trial163.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T10:36:20.377 INFO:teuthology.orchestra.run.trial163.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T10:36:20.378 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:20.386 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 41/46 2026-02-13T10:36:20.409 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 41/46 2026-02-13T10:36:20.409 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T10:36:20.409 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:20.446 INFO:teuthology.orchestra.run.trial121.stdout:(20/46): python3-markupsafe-1.1.1-12.el9.x86_64 220 kB/s | 35 kB 00:00 2026-02-13T10:36:20.595 INFO:teuthology.orchestra.run.trial121.stdout:(21/46): python3-libstoragemgmt-1.10.1-1.el9.x8 224 kB/s | 177 kB 00:00 2026-02-13T10:36:20.618 INFO:teuthology.orchestra.run.trial121.stdout:(22/46): python3-packaging-20.9-5.el9.noarch.rp 449 kB/s | 77 kB 00:00 2026-02-13T10:36:20.753 INFO:teuthology.orchestra.run.trial121.stdout:(23/46): python3-toml-0.10.2-6.el9.noarch.rpm 264 kB/s | 42 kB 00:00 2026-02-13T10:36:20.801 INFO:teuthology.orchestra.run.trial121.stdout:(24/46): lua-5.4.4-4.el9.x86_64.rpm 172 kB/s | 188 kB 00:01 2026-02-13T10:36:20.853 INFO:teuthology.orchestra.run.trial121.stdout:(25/46): lua-devel-5.4.4-4.el9.x86_64.rpm 95 kB/s | 22 kB 00:00 2026-02-13T10:36:20.967 INFO:teuthology.orchestra.run.trial121.stdout:(26/46): python3-autocommand-2.2.2-8.el9.noarch 177 kB/s | 29 kB 00:00 2026-02-13T10:36:21.009 INFO:teuthology.orchestra.run.trial121.stdout:(27/46): luarocks-3.9.2-5.el9.noarch.rpm 590 kB/s | 151 kB 00:00 2026-02-13T10:36:21.034 INFO:teuthology.orchestra.run.trial121.stdout:(28/46): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 647 kB/s | 43 kB 00:00 2026-02-13T10:36:21.051 INFO:teuthology.orchestra.run.trial121.stdout:(29/46): python3-backports-tarfile-1.2.0-1.el9. 305 kB/s | 60 kB 00:00 2026-02-13T10:36:21.075 INFO:teuthology.orchestra.run.trial121.stdout:(30/46): python3-cheroot-10.0.1-4.el9.noarch.rp 2.6 MB/s | 173 kB 00:00 2026-02-13T10:36:21.091 INFO:teuthology.orchestra.run.trial121.stdout:(31/46): python3-jaraco-8.2.1-3.el9.noarch.rpm 265 kB/s | 11 kB 00:00 2026-02-13T10:36:21.116 INFO:teuthology.orchestra.run.trial121.stdout:(32/46): python3-jaraco-classes-3.2.1-5.el9.noa 433 kB/s | 18 kB 00:00 2026-02-13T10:36:21.136 INFO:teuthology.orchestra.run.trial121.stdout:(33/46): python3-jaraco-collections-3.0.0-8.el9 513 kB/s | 23 kB 00:00 2026-02-13T10:36:21.137 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 42/46 2026-02-13T10:36:21.139 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 43/46 2026-02-13T10:36:21.155 INFO:teuthology.orchestra.run.trial121.stdout:(34/46): python3-jaraco-context-6.0.1-3.el9.noa 503 kB/s | 20 kB 00:00 2026-02-13T10:36:21.164 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 43/46 2026-02-13T10:36:21.164 INFO:teuthology.orchestra.run.trial163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:21.164 INFO:teuthology.orchestra.run.trial163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T10:36:21.164 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T10:36:21.164 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T10:36:21.164 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:21.180 INFO:teuthology.orchestra.run.trial121.stdout:(35/46): python3-jaraco-functools-3.5.0-2.el9.n 449 kB/s | 19 kB 00:00 2026-02-13T10:36:21.187 INFO:teuthology.orchestra.run.trial121.stdout:(36/46): python3-cherrypy-18.6.1-2.el9.noarch.r 2.3 MB/s | 358 kB 00:00 2026-02-13T10:36:21.195 INFO:teuthology.orchestra.run.trial121.stdout:(37/46): python3-jaraco-text-4.0.0-2.el9.noarch 661 kB/s | 26 kB 00:00 2026-02-13T10:36:21.233 INFO:teuthology.orchestra.run.trial121.stdout:(38/46): python3-more-itertools-8.12.0-2.el9.no 1.7 MB/s | 79 kB 00:00 2026-02-13T10:36:21.243 INFO:teuthology.orchestra.run.trial121.stdout:(39/46): python3-logutils-0.3.5-21.el9.noarch.r 726 kB/s | 46 kB 00:00 2026-02-13T10:36:21.271 INFO:teuthology.orchestra.run.trial121.stdout:(40/46): python3-portend-3.1.0-2.el9.noarch.rpm 432 kB/s | 16 kB 00:00 2026-02-13T10:36:21.272 INFO:teuthology.orchestra.run.trial121.stdout:(41/46): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2026-02-13T10:36:21.310 INFO:teuthology.orchestra.run.trial121.stdout:(42/46): python3-tempora-5.0.0-2.el9.noarch.rpm 918 kB/s | 36 kB 00:00 2026-02-13T10:36:21.312 INFO:teuthology.orchestra.run.trial121.stdout:(43/46): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2026-02-13T10:36:21.334 INFO:teuthology.orchestra.run.trial121.stdout:(44/46): python3-webob-1.8.8-2.el9.noarch.rpm 3.6 MB/s | 230 kB 00:00 2026-02-13T10:36:21.360 INFO:teuthology.orchestra.run.trial121.stdout:(45/46): python3-zc-lockfile-2.0-10.el9.noarch. 416 kB/s | 20 kB 00:00 2026-02-13T10:36:21.371 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 44/46 2026-02-13T10:36:21.382 INFO:teuthology.orchestra.run.trial121.stdout:(46/46): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.8 MB/s | 427 kB 00:00 2026-02-13T10:36:21.385 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:21.385 INFO:teuthology.orchestra.run.trial121.stdout:Total 6.1 MB/s | 31 MB 00:05 2026-02-13T10:36:21.392 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 44/46 2026-02-13T10:36:21.392 INFO:teuthology.orchestra.run.trial163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:21.392 INFO:teuthology.orchestra.run.trial163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T10:36:21.392 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T10:36:21.392 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T10:36:21.392 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:21.495 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 45/46 2026-02-13T10:36:21.514 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 45/46 2026-02-13T10:36:21.515 INFO:teuthology.orchestra.run.trial163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:21.515 INFO:teuthology.orchestra.run.trial163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T10:36:21.515 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T10:36:21.515 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T10:36:21.515 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:21.531 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-2:19.2.3-672.g283431c0.el9.x86_64 46/46 2026-02-13T10:36:21.544 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:21.563 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:21.563 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:21.716 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:21.717 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:21.872 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-2:19.2.3-672.g283431c0.el9.x86_64 46/46 2026-02-13T10:36:21.872 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-2:19.2.3-672.g283431c0.el9.x86_64 1/46 2026-02-13T10:36:21.872 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 2/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 3/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 4/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 5/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-672.g283431c0.el9.n 6/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 7/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 8/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 9/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 10/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 13/46 2026-02-13T10:36:21.873 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 14/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 15/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : unzip-6.0-59.el9.x86_64 16/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : zip-3.0-35.el9.x86_64 17/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 18/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : lua-5.4.4-4.el9.x86_64 19/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 20/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 21/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 22/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/46 2026-02-13T10:36:21.874 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 38/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 39/46 2026-02-13T10:36:21.875 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 40/46 2026-02-13T10:36:21.876 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 41/46 2026-02-13T10:36:21.876 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 42/46 2026-02-13T10:36:21.876 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 43/46 2026-02-13T10:36:21.876 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 44/46 2026-02-13T10:36:21.876 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 45/46 2026-02-13T10:36:21.905 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:21.907 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/46 2026-02-13T10:36:21.911 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/46 2026-02-13T10:36:21.916 INFO:teuthology.orchestra.run.trial121.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/46 2026-02-13T10:36:21.918 INFO:teuthology.orchestra.run.trial121.stdout: Installing : unzip-6.0-59.el9.x86_64 4/46 2026-02-13T10:36:21.924 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/46 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 46/46 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: ceph-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-modules-core-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:21.928 INFO:teuthology.orchestra.run.trial163.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T10:36:21.929 INFO:teuthology.orchestra.run.trial163.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T10:36:21.930 INFO:teuthology.orchestra.run.trial163.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-13T10:36:21.931 INFO:teuthology.orchestra.run.trial163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T10:36:21.932 INFO:teuthology.orchestra.run.trial163.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T10:36:21.932 INFO:teuthology.orchestra.run.trial163.stdout: zip-3.0-35.el9.x86_64 2026-02-13T10:36:21.932 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:21.932 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:21.948 INFO:teuthology.orchestra.run.trial121.stdout: Installing : zip-3.0-35.el9.x86_64 6/46 2026-02-13T10:36:21.951 INFO:teuthology.orchestra.run.trial121.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/46 2026-02-13T10:36:21.956 INFO:teuthology.orchestra.run.trial121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/46 2026-02-13T10:36:21.961 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/46 2026-02-13T10:36:21.965 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-base 2026-02-13T10:36:21.992 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/46 2026-02-13T10:36:22.013 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 11/46 2026-02-13T10:36:22.022 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 12/46 2026-02-13T10:36:22.029 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 13/46 2026-02-13T10:36:22.032 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 14/46 2026-02-13T10:36:22.038 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 15/46 2026-02-13T10:36:22.044 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 16/46 2026-02-13T10:36:22.047 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/46 2026-02-13T10:36:22.053 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/46 2026-02-13T10:36:22.056 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 19/46 2026-02-13T10:36:22.063 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/46 2026-02-13T10:36:22.066 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 21/46 2026-02-13T10:36:22.072 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 22/46 2026-02-13T10:36:22.096 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/46 2026-02-13T10:36:22.151 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 24/46 2026-02-13T10:36:22.175 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 25/46 2026-02-13T10:36:22.188 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 26/46 2026-02-13T10:36:22.193 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 27/46 2026-02-13T10:36:22.202 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 28/46 2026-02-13T10:36:22.209 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 29/46 2026-02-13T10:36:22.222 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-ply-3.11-14.el9.noarch 30/46 2026-02-13T10:36:22.238 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 31/46 2026-02-13T10:36:22.269 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:22 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:22.313 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 32/46 2026-02-13T10:36:22.321 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 33/46 2026-02-13T10:36:22.333 INFO:teuthology.orchestra.run.trial163.stdout:Package ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:22.345 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 34/46 2026-02-13T10:36:22.351 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:22.352 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:22.352 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:22.373 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install cephadm 2026-02-13T10:36:22.374 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 35/46 2026-02-13T10:36:22.380 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 36/46 2026-02-13T10:36:22.426 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 37/46 2026-02-13T10:36:22.475 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-672.g283431c0.el9.n 38/46 2026-02-13T10:36:22.476 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 39/46 2026-02-13T10:36:22.499 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 39/46 2026-02-13T10:36:22.499 INFO:teuthology.orchestra.run.trial121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:22.499 INFO:teuthology.orchestra.run.trial121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T10:36:22.499 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T10:36:22.499 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T10:36:22.499 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:22.503 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 40/46 2026-02-13T10:36:22.519 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 41/46 2026-02-13T10:36:22.519 INFO:teuthology.orchestra.run.trial121.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T10:36:22.520 INFO:teuthology.orchestra.run.trial121.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T10:36:22.520 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:22.527 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 41/46 2026-02-13T10:36:22.550 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 41/46 2026-02-13T10:36:22.550 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T10:36:22.551 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:22.666 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:22 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:22.748 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout: cephadm noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 769 k 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:22.749 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:22.750 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 769 k 2026-02-13T10:36:22.750 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 775 k 2026-02-13T10:36:22.750 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:22.867 INFO:teuthology.orchestra.run.trial163.stdout:cephadm-19.2.3-672.g283431c0.el9.noarch.rpm 6.4 MB/s | 769 kB 00:00 2026-02-13T10:36:22.868 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:22.868 INFO:teuthology.orchestra.run.trial163.stdout:Total 6.3 MB/s | 769 kB 00:00 2026-02-13T10:36:22.868 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:22.871 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:22.871 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:22.874 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:22.874 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:22.892 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:22.948 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:22.962 INFO:teuthology.orchestra.run.trial163.stdout: Installing : cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:23.024 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:23.077 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:23.077 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:23.077 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:23.077 INFO:teuthology.orchestra.run.trial163.stdout: cephadm-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:23.077 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:23.077 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:23.103 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-immutable-object-cache 2026-02-13T10:36:23.263 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 42/46 2026-02-13T10:36:23.265 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 43/46 2026-02-13T10:36:23.289 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 43/46 2026-02-13T10:36:23.289 INFO:teuthology.orchestra.run.trial121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:23.290 INFO:teuthology.orchestra.run.trial121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T10:36:23.290 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T10:36:23.290 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T10:36:23.290 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:23.393 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:23 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:23.475 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:23.476 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:23.476 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repo Size 2026-02-13T10:36:23.476 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:23.476 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-672.g283431c0.el9 ceph 145 k 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 145 k 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 435 k 2026-02-13T10:36:23.477 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:23.493 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 44/46 2026-02-13T10:36:23.515 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 44/46 2026-02-13T10:36:23.515 INFO:teuthology.orchestra.run.trial121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:23.515 INFO:teuthology.orchestra.run.trial121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T10:36:23.515 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T10:36:23.515 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T10:36:23.516 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:23.571 INFO:teuthology.orchestra.run.trial163.stdout:ceph-immutable-object-cache-19.2.3-672.g283431c 1.5 MB/s | 145 kB 00:00 2026-02-13T10:36:23.571 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:23.572 INFO:teuthology.orchestra.run.trial163.stdout:Total 1.5 MB/s | 145 kB 00:00 2026-02-13T10:36:23.572 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:23.575 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:23.575 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:23.601 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:23.601 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:23.616 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 45/46 2026-02-13T10:36:23.636 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 45/46 2026-02-13T10:36:23.636 INFO:teuthology.orchestra.run.trial121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:23.637 INFO:teuthology.orchestra.run.trial121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T10:36:23.637 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T10:36:23.637 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T10:36:23.637 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:23.650 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:23.651 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-672.g283431c0.e 1/1 2026-02-13T10:36:23.653 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-2:19.2.3-672.g283431c0.el9.x86_64 46/46 2026-02-13T10:36:23.674 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-672.g283431c0.e 1/1 2026-02-13T10:36:23.674 INFO:teuthology.orchestra.run.trial163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:23.674 INFO:teuthology.orchestra.run.trial163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T10:36:23.674 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:23.974 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-672.g283431c0.e 1/1 2026-02-13T10:36:23.974 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:23.974 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:23.974 INFO:teuthology.orchestra.run.trial163.stdout: ceph-immutable-object-cache-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:23.974 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:23.974 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-2:19.2.3-672.g283431c0.el9.x86_64 46/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-2:19.2.3-672.g283431c0.el9.x86_64 1/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 2/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 3/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 4/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 5/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-672.g283431c0.el9.n 6/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 7/46 2026-02-13T10:36:23.982 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 8/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 9/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 10/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 13/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 14/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 15/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : unzip-6.0-59.el9.x86_64 16/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : zip-3.0-35.el9.x86_64 17/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 18/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 19/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 20/46 2026-02-13T10:36:23.983 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 21/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 22/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/46 2026-02-13T10:36:23.984 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 38/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 39/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 40/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 41/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 42/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 43/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 44/46 2026-02-13T10:36:23.985 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 45/46 2026-02-13T10:36:24.002 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-mgr 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 46/46 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: ceph-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mds-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-modules-core-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mon-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:24.038 INFO:teuthology.orchestra.run.trial121.stdout: ceph-osd-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T10:36:24.039 INFO:teuthology.orchestra.run.trial121.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T10:36:24.040 INFO:teuthology.orchestra.run.trial121.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-13T10:36:24.041 INFO:teuthology.orchestra.run.trial121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T10:36:24.042 INFO:teuthology.orchestra.run.trial121.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T10:36:24.042 INFO:teuthology.orchestra.run.trial121.stdout: zip-3.0-35.el9.x86_64 2026-02-13T10:36:24.042 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:24.042 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:24.074 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-base 2026-02-13T10:36:24.297 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:24 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:24.361 INFO:teuthology.orchestra.run.trial163.stdout:Package ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:24.373 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:24.379 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:24.380 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:24.380 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:24.402 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-mgr-dashboard 2026-02-13T10:36:24.437 INFO:teuthology.orchestra.run.trial121.stdout:Package ceph-base-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:24.455 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:24.456 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:24.456 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:24.477 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install cephadm 2026-02-13T10:36:24.696 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:24 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:24.776 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:24.778 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-dashboard noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 3.8 M 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout: ceph-grafana-dashboards noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 31 k 2026-02-13T10:36:24.779 INFO:teuthology.orchestra.run.trial163.stdout: ceph-prometheus-alerts noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 17 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:24.780 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:24.781 INFO:teuthology.orchestra.run.trial163.stdout:Install 13 Packages 2026-02-13T10:36:24.781 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:24.781 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 8.9 M 2026-02-13T10:36:24.781 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 106 M 2026-02-13T10:36:24.782 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:24.860 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout: cephadm noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 769 k 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:24.861 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:24.862 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 769 k 2026-02-13T10:36:24.862 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 775 k 2026-02-13T10:36:24.862 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:24.970 INFO:teuthology.orchestra.run.trial121.stdout:cephadm-19.2.3-672.g283431c0.el9.noarch.rpm 6.9 MB/s | 769 kB 00:00 2026-02-13T10:36:24.970 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:24.970 INFO:teuthology.orchestra.run.trial121.stdout:Total 6.9 MB/s | 769 kB 00:00 2026-02-13T10:36:24.971 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:24.973 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:24.973 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:24.976 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:24.976 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:24.994 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:25.050 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:25.064 INFO:teuthology.orchestra.run.trial121.stdout: Installing : cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:25.129 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:25.184 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/1 2026-02-13T10:36:25.184 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:25.184 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:25.184 INFO:teuthology.orchestra.run.trial121.stdout: cephadm-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:25.184 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:25.184 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:25.185 INFO:teuthology.orchestra.run.trial163.stdout:(1/13): ceph-grafana-dashboards-19.2.3-672.g283 505 kB/s | 31 kB 00:00 2026-02-13T10:36:25.192 INFO:teuthology.orchestra.run.trial163.stdout:(2/13): ceph-prometheus-alerts-19.2.3-672.g2834 246 kB/s | 17 kB 00:00 2026-02-13T10:36:25.213 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-immutable-object-cache 2026-02-13T10:36:25.248 INFO:teuthology.orchestra.run.trial163.stdout:(3/13): python3-protobuf-3.14.0-17.el9.noarch.r 4.7 MB/s | 267 kB 00:00 2026-02-13T10:36:25.249 INFO:teuthology.orchestra.run.trial163.stdout:(4/13): protobuf-3.14.0-17.el9.x86_64.rpm 16 MB/s | 1.0 MB 00:00 2026-02-13T10:36:25.288 INFO:teuthology.orchestra.run.trial163.stdout:(5/13): ceph-mgr-dashboard-19.2.3-672.g283431c0 23 MB/s | 3.8 MB 00:00 2026-02-13T10:36:25.378 INFO:teuthology.orchestra.run.trial163.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 218 kB/s | 19 kB 00:00 2026-02-13T10:36:25.496 INFO:teuthology.orchestra.run.trial163.stdout:(7/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.2 MB/s | 551 kB 00:00 2026-02-13T10:36:25.508 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:28 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:25.532 INFO:teuthology.orchestra.run.trial163.stdout:(8/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.0 MB/s | 144 kB 00:00 2026-02-13T10:36:25.565 INFO:teuthology.orchestra.run.trial163.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 914 kB/s | 31 kB 00:00 2026-02-13T10:36:25.590 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:25.590 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repo Size 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-672.g283431c0.el9 ceph 145 k 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:25.591 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:25.592 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 145 k 2026-02-13T10:36:25.592 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 435 k 2026-02-13T10:36:25.592 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:25.600 INFO:teuthology.orchestra.run.trial163.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 5.3 MB/s | 188 kB 00:00 2026-02-13T10:36:25.634 INFO:teuthology.orchestra.run.trial163.stdout:(11/13): python3-xmltodict-0.12.0-15.el9.noarch 667 kB/s | 22 kB 00:00 2026-02-13T10:36:25.644 INFO:teuthology.orchestra.run.trial163.stdout:(12/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 7.6 MB/s | 2.0 MB 00:00 2026-02-13T10:36:25.675 INFO:teuthology.orchestra.run.trial121.stdout:ceph-immutable-object-cache-19.2.3-672.g283431c 1.7 MB/s | 145 kB 00:00 2026-02-13T10:36:25.676 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:25.676 INFO:teuthology.orchestra.run.trial121.stdout:Total 1.7 MB/s | 145 kB 00:00 2026-02-13T10:36:25.676 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:25.680 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:25.680 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:25.706 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:25.707 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:25.755 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:25.757 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-672.g283431c0.e 1/1 2026-02-13T10:36:25.779 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-672.g283431c0.e 1/1 2026-02-13T10:36:25.779 INFO:teuthology.orchestra.run.trial121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:25.779 INFO:teuthology.orchestra.run.trial121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T10:36:25.779 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:26.054 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-672.g283431c0.e 1/1 2026-02-13T10:36:26.054 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:26.054 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:26.054 INFO:teuthology.orchestra.run.trial121.stdout: ceph-immutable-object-cache-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:26.054 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:26.054 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:26.082 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-mgr 2026-02-13T10:36:26.281 INFO:teuthology.orchestra.run.trial163.stdout:(13/13): protobuf-compiler-3.14.0-17.el9.x86_64 834 kB/s | 862 kB 00:01 2026-02-13T10:36:26.286 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:26.286 INFO:teuthology.orchestra.run.trial163.stdout:Total 5.9 MB/s | 8.9 MB 00:01 2026-02-13T10:36:26.315 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:26.324 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:26.324 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:26.384 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:29 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:26.399 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:26.400 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:26.449 INFO:teuthology.orchestra.run.trial121.stdout:Package ceph-mgr-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:26.467 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:26.467 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:26.467 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:26.489 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-mgr-dashboard 2026-02-13T10:36:26.495 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:26.503 INFO:teuthology.orchestra.run.trial163.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 1/13 2026-02-13T10:36:26.506 INFO:teuthology.orchestra.run.trial163.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 2/13 2026-02-13T10:36:26.511 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 3/13 2026-02-13T10:36:26.522 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 4/13 2026-02-13T10:36:26.525 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 5/13 2026-02-13T10:36:26.552 INFO:teuthology.orchestra.run.trial163.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 6/13 2026-02-13T10:36:26.596 INFO:teuthology.orchestra.run.trial163.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-13T10:36:26.625 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 8/13 2026-02-13T10:36:26.635 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 9/13 2026-02-13T10:36:26.638 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-13T10:36:26.643 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-672.g283431c0.el9. 11/13 2026-02-13T10:36:26.785 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:29 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:26.867 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:26.868 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:26.868 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:36:26.868 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-dashboard noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 3.8 M 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: ceph-grafana-dashboards noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 31 k 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: ceph-prometheus-alerts noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 17 k 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T10:36:26.869 INFO:teuthology.orchestra.run.trial121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout:Install 13 Packages 2026-02-13T10:36:26.870 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:26.871 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 8.9 M 2026-02-13T10:36:26.871 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 106 M 2026-02-13T10:36:26.871 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:27.122 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-672.g283431c0.el9 12/13 2026-02-13T10:36:27.125 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noar 13/13 2026-02-13T10:36:27.133 INFO:teuthology.orchestra.run.trial121.stdout:(1/13): ceph-grafana-dashboards-19.2.3-672.g283 513 kB/s | 31 kB 00:00 2026-02-13T10:36:27.141 INFO:teuthology.orchestra.run.trial121.stdout:(2/13): ceph-prometheus-alerts-19.2.3-672.g2834 247 kB/s | 17 kB 00:00 2026-02-13T10:36:27.236 INFO:teuthology.orchestra.run.trial121.stdout:(3/13): ceph-mgr-dashboard-19.2.3-672.g283431c0 23 MB/s | 3.8 MB 00:00 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noar 13/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-672.g283431c0.el9 1/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noar 2/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-672.g283431c0.el9. 3/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 4/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-13T10:36:27.294 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-13T10:36:27.295 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-13T10:36:27.295 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-13T10:36:27.295 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-13T10:36:27.295 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-13T10:36:27.338 INFO:teuthology.orchestra.run.trial121.stdout:(4/13): python3-protobuf-3.14.0-17.el9.noarch.r 1.3 MB/s | 267 kB 00:00 2026-02-13T10:36:27.348 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-13T10:36:27.348 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:27.348 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:27.348 INFO:teuthology.orchestra.run.trial163.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T10:36:27.348 INFO:teuthology.orchestra.run.trial163.stdout: ceph-grafana-dashboards-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:27.348 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: ceph-prometheus-alerts-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:27.349 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:27.378 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-13T10:36:27.386 INFO:teuthology.orchestra.run.trial121.stdout:(5/13): protobuf-3.14.0-17.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2026-02-13T10:36:27.542 INFO:teuthology.orchestra.run.trial121.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 124 kB/s | 19 kB 00:00 2026-02-13T10:36:27.656 INFO:teuthology.orchestra.run.trial121.stdout:(7/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.7 MB/s | 551 kB 00:00 2026-02-13T10:36:27.678 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:27.709 INFO:teuthology.orchestra.run.trial121.stdout:(8/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.7 MB/s | 144 kB 00:00 2026-02-13T10:36:27.751 INFO:teuthology.orchestra.run.trial121.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 722 kB/s | 31 kB 00:00 2026-02-13T10:36:27.760 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:27.761 INFO:teuthology.orchestra.run.trial163.stdout:====================================================================================== 2026-02-13T10:36:27.761 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:27.761 INFO:teuthology.orchestra.run.trial163.stdout:====================================================================================== 2026-02-13T10:36:27.761 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 7.4 M 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T10:36:27.762 INFO:teuthology.orchestra.run.trial163.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T10:36:27.763 INFO:teuthology.orchestra.run.trial163.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T10:36:27.763 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:27.763 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:27.763 INFO:teuthology.orchestra.run.trial163.stdout:====================================================================================== 2026-02-13T10:36:27.763 INFO:teuthology.orchestra.run.trial163.stdout:Install 12 Packages 2026-02-13T10:36:27.763 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:27.764 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 43 M 2026-02-13T10:36:27.764 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 234 M 2026-02-13T10:36:27.764 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:27.806 INFO:teuthology.orchestra.run.trial121.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 3.4 MB/s | 188 kB 00:00 2026-02-13T10:36:27.812 INFO:teuthology.orchestra.run.trial121.stdout:(11/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 7.5 MB/s | 2.0 MB 00:00 2026-02-13T10:36:27.855 INFO:teuthology.orchestra.run.trial121.stdout:(12/13): python3-xmltodict-0.12.0-15.el9.noarch 458 kB/s | 22 kB 00:00 2026-02-13T10:36:27.990 INFO:teuthology.orchestra.run.trial121.stdout:(13/13): protobuf-compiler-3.14.0-17.el9.x86_64 1.1 MB/s | 862 kB 00:00 2026-02-13T10:36:27.993 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:27.993 INFO:teuthology.orchestra.run.trial121.stdout:Total 7.9 MB/s | 8.9 MB 00:01 2026-02-13T10:36:28.021 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:28.029 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:28.029 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:28.102 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:28.102 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:28.196 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:28.204 INFO:teuthology.orchestra.run.trial121.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 1/13 2026-02-13T10:36:28.206 INFO:teuthology.orchestra.run.trial163.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 3.5 MB/s | 184 kB 00:00 2026-02-13T10:36:28.208 INFO:teuthology.orchestra.run.trial121.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 2/13 2026-02-13T10:36:28.212 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 3/13 2026-02-13T10:36:28.224 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 4/13 2026-02-13T10:36:28.226 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 5/13 2026-02-13T10:36:28.231 INFO:teuthology.orchestra.run.trial163.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 33 kB 00:00 2026-02-13T10:36:28.253 INFO:teuthology.orchestra.run.trial121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 6/13 2026-02-13T10:36:28.291 INFO:teuthology.orchestra.run.trial163.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 50 MB/s | 3.0 MB 00:00 2026-02-13T10:36:28.295 INFO:teuthology.orchestra.run.trial121.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-13T10:36:28.300 INFO:teuthology.orchestra.run.trial163.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 2.1 MB/s | 18 kB 00:00 2026-02-13T10:36:28.308 INFO:teuthology.orchestra.run.trial163.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 4.9 MB/s | 42 kB 00:00 2026-02-13T10:36:28.323 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 8/13 2026-02-13T10:36:28.333 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 9/13 2026-02-13T10:36:28.336 INFO:teuthology.orchestra.run.trial163.stdout:(6/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 193 MB/s | 5.3 MB 00:00 2026-02-13T10:36:28.336 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-13T10:36:28.340 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-672.g283431c0.el9. 11/13 2026-02-13T10:36:28.347 INFO:teuthology.orchestra.run.trial163.stdout:(7/12): python3-devel-3.9.25-2.el9.x86_64.rpm 21 MB/s | 244 kB 00:00 2026-02-13T10:36:28.363 INFO:teuthology.orchestra.run.trial163.stdout:(8/12): libgfortran-11.5.0-14.el9.x86_64.rpm 3.7 MB/s | 794 kB 00:00 2026-02-13T10:36:28.381 INFO:teuthology.orchestra.run.trial163.stdout:(9/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 181 MB/s | 6.1 MB 00:00 2026-02-13T10:36:28.391 INFO:teuthology.orchestra.run.trial163.stdout:(10/12): ceph-mgr-diskprediction-local-19.2.3-6 31 MB/s | 7.4 MB 00:00 2026-02-13T10:36:28.416 INFO:teuthology.orchestra.run.trial163.stdout:(11/12): python3-numpy-f2py-1.23.5-2.el9.x86_64 8.1 MB/s | 442 kB 00:00 2026-02-13T10:36:28.476 INFO:teuthology.orchestra.run.trial163.stdout:(12/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 203 MB/s | 19 MB 00:00 2026-02-13T10:36:28.478 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:28.478 INFO:teuthology.orchestra.run.trial163.stdout:Total 60 MB/s | 43 MB 00:00 2026-02-13T10:36:28.644 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:28.657 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:28.658 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:28.820 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-672.g283431c0.el9 12/13 2026-02-13T10:36:28.822 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noar 13/13 2026-02-13T10:36:28.860 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:28.861 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:28.969 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noar 13/13 2026-02-13T10:36:28.969 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-672.g283431c0.el9 1/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noar 2/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-672.g283431c0.el9. 3/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 4/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-13T10:36:28.970 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-13T10:36:29.023 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-13T10:36:29.023 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:29.023 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:29.023 INFO:teuthology.orchestra.run.trial121.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T10:36:29.023 INFO:teuthology.orchestra.run.trial121.stdout: ceph-grafana-dashboards-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:29.023 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-dashboard-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:29.023 INFO:teuthology.orchestra.run.trial121.stdout: ceph-prometheus-alerts-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:29.024 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:29.054 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-13T10:36:29.080 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:29.083 INFO:teuthology.orchestra.run.trial163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2026-02-13T10:36:29.089 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-13T10:36:29.122 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-13T10:36:29.127 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 4/12 2026-02-13T10:36:29.186 INFO:teuthology.orchestra.run.trial163.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-13T10:36:29.188 INFO:teuthology.orchestra.run.trial163.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-13T10:36:29.209 INFO:teuthology.orchestra.run.trial163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2026-02-13T10:36:29.354 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:32 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:29.437 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:29.437 INFO:teuthology.orchestra.run.trial121.stdout:====================================================================================== 2026-02-13T10:36:29.437 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout:====================================================================================== 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 7.4 M 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T10:36:29.438 INFO:teuthology.orchestra.run.trial121.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout:====================================================================================== 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout:Install 12 Packages 2026-02-13T10:36:29.439 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:29.440 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 43 M 2026-02-13T10:36:29.440 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 234 M 2026-02-13T10:36:29.440 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:29.530 INFO:teuthology.orchestra.run.trial163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2026-02-13T10:36:29.602 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-13T10:36:29.660 INFO:teuthology.orchestra.run.trial121.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 4.1 MB/s | 184 kB 00:00 2026-02-13T10:36:29.684 INFO:teuthology.orchestra.run.trial121.stdout:(2/12): libgfortran-11.5.0-14.el9.x86_64.rpm 11 MB/s | 794 kB 00:00 2026-02-13T10:36:29.728 INFO:teuthology.orchestra.run.trial121.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 485 kB/s | 33 kB 00:00 2026-02-13T10:36:29.750 INFO:teuthology.orchestra.run.trial121.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 835 kB/s | 18 kB 00:00 2026-02-13T10:36:29.772 INFO:teuthology.orchestra.run.trial121.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-13T10:36:29.842 INFO:teuthology.orchestra.run.trial121.stdout:(6/12): ceph-mgr-diskprediction-local-19.2.3-67 33 MB/s | 7.4 MB 00:00 2026-02-13T10:36:29.855 INFO:teuthology.orchestra.run.trial121.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2026-02-13T10:36:29.905 INFO:teuthology.orchestra.run.trial121.stdout:(8/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 40 MB/s | 5.3 MB 00:00 2026-02-13T10:36:29.917 INFO:teuthology.orchestra.run.trial121.stdout:(9/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 98 MB/s | 6.1 MB 00:00 2026-02-13T10:36:29.929 INFO:teuthology.orchestra.run.trial121.stdout:(10/12): python3-devel-3.9.25-2.el9.x86_64.rpm 2.8 MB/s | 244 kB 00:00 2026-02-13T10:36:29.931 INFO:teuthology.orchestra.run.trial121.stdout:(11/12): python3-numpy-f2py-1.23.5-2.el9.x86_64 17 MB/s | 442 kB 00:00 2026-02-13T10:36:30.069 INFO:teuthology.orchestra.run.trial121.stdout:(12/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 127 MB/s | 19 MB 00:00 2026-02-13T10:36:30.070 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:30.070 INFO:teuthology.orchestra.run.trial121.stdout:Total 68 MB/s | 43 MB 00:00 2026-02-13T10:36:30.232 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:30.245 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:30.246 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:30.263 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T10:36:30.448 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:30.448 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:30.665 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:30.668 INFO:teuthology.orchestra.run.trial121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2026-02-13T10:36:30.674 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-13T10:36:30.707 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-13T10:36:30.712 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 4/12 2026-02-13T10:36:30.723 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-13T10:36:30.724 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-672.g283431 12/12 2026-02-13T10:36:30.771 INFO:teuthology.orchestra.run.trial121.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-13T10:36:30.773 INFO:teuthology.orchestra.run.trial121.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-13T10:36:30.794 INFO:teuthology.orchestra.run.trial121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2026-02-13T10:36:30.997 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-672.g283431 12/12 2026-02-13T10:36:30.997 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-672.g283431 1/12 2026-02-13T10:36:30.997 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 9/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T10:36:30.998 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-13T10:36:31.053 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-13T10:36:31.053 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-diskprediction-local-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T10:36:31.054 INFO:teuthology.orchestra.run.trial163.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T10:36:31.055 INFO:teuthology.orchestra.run.trial163.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T10:36:31.055 INFO:teuthology.orchestra.run.trial163.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T10:36:31.055 INFO:teuthology.orchestra.run.trial163.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T10:36:31.055 INFO:teuthology.orchestra.run.trial163.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T10:36:31.055 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:31.055 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:31.092 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-mgr-rook 2026-02-13T10:36:31.113 INFO:teuthology.orchestra.run.trial121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2026-02-13T10:36:31.184 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-13T10:36:31.388 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:31 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:31.472 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:31.472 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:31.472 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-rook noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 49 k 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T10:36:31.473 INFO:teuthology.orchestra.run.trial163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T10:36:31.474 INFO:teuthology.orchestra.run.trial163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T10:36:31.474 INFO:teuthology.orchestra.run.trial163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T10:36:31.474 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:31.474 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:31.474 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:31.474 INFO:teuthology.orchestra.run.trial163.stdout:Install 10 Packages 2026-02-13T10:36:31.474 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:31.475 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 2.0 M 2026-02-13T10:36:31.475 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 25 M 2026-02-13T10:36:31.475 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:31.714 INFO:teuthology.orchestra.run.trial163.stdout:(1/10): ceph-mgr-rook-19.2.3-672.g283431c0.el9. 679 kB/s | 49 kB 00:00 2026-02-13T10:36:31.842 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T10:36:31.957 INFO:teuthology.orchestra.run.trial163.stdout:(2/10): python3-requests-oauthlib-1.3.0-12.el9. 221 kB/s | 54 kB 00:00 2026-02-13T10:36:32.004 INFO:teuthology.orchestra.run.trial163.stdout:(3/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 438 kB/s | 159 kB 00:00 2026-02-13T10:36:32.081 INFO:teuthology.orchestra.run.trial163.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 258 kB/s | 32 kB 00:00 2026-02-13T10:36:32.084 INFO:teuthology.orchestra.run.trial163.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 177 kB/s | 14 kB 00:00 2026-02-13T10:36:32.094 INFO:teuthology.orchestra.run.trial163.stdout:(6/10): python3-pyasn1-modules-0.4.8-6.el9.noar 616 kB/s | 279 kB 00:00 2026-02-13T10:36:32.193 INFO:teuthology.orchestra.run.trial163.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 597 kB/s | 59 kB 00:00 2026-02-13T10:36:32.300 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-13T10:36:32.301 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-672.g283431 12/12 2026-02-13T10:36:32.576 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-672.g283431 12/12 2026-02-13T10:36:32.576 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-672.g283431 1/12 2026-02-13T10:36:32.576 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-13T10:36:32.576 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-13T10:36:32.576 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2026-02-13T10:36:32.577 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2026-02-13T10:36:32.577 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2026-02-13T10:36:32.577 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-13T10:36:32.577 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-13T10:36:32.577 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 9/12 2026-02-13T10:36:32.577 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-13T10:36:32.577 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-13T10:36:32.634 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-13T10:36:32.634 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:32.634 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:32.634 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-diskprediction-local-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:32.634 INFO:teuthology.orchestra.run.trial121.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:32.635 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:32.673 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-mgr-rook 2026-02-13T10:36:32.977 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:35 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:33.060 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:33.061 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:33.061 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:36:33.061 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:33.061 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:33.061 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-rook noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 49 k 2026-02-13T10:36:33.061 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:33.062 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:33.063 INFO:teuthology.orchestra.run.trial121.stdout:Install 10 Packages 2026-02-13T10:36:33.063 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:33.063 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 2.0 M 2026-02-13T10:36:33.063 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 25 M 2026-02-13T10:36:33.064 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:33.275 INFO:teuthology.orchestra.run.trial121.stdout:(1/10): ceph-mgr-rook-19.2.3-672.g283431c0.el9. 674 kB/s | 49 kB 00:00 2026-02-13T10:36:33.912 INFO:teuthology.orchestra.run.trial121.stdout:(2/10): python3-requests-oauthlib-1.3.0-12.el9. 84 kB/s | 54 kB 00:00 2026-02-13T10:36:34.013 INFO:teuthology.orchestra.run.trial121.stdout:(3/10): python3-pyasn1-modules-0.4.8-6.el9.noar 344 kB/s | 279 kB 00:00 2026-02-13T10:36:34.122 INFO:teuthology.orchestra.run.trial121.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 153 kB/s | 32 kB 00:00 2026-02-13T10:36:34.200 INFO:teuthology.orchestra.run.trial121.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 76 kB/s | 14 kB 00:00 2026-02-13T10:36:34.224 INFO:teuthology.orchestra.run.trial121.stdout:(6/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 155 kB/s | 159 kB 00:01 2026-02-13T10:36:34.306 INFO:teuthology.orchestra.run.trial121.stdout:(7/10): python3-google-auth-2.45.0-1.el9.noarch 1.4 MB/s | 254 kB 00:00 2026-02-13T10:36:34.361 INFO:teuthology.orchestra.run.trial121.stdout:(8/10): python3-websocket-client-1.2.3-2.el9.no 1.6 MB/s | 90 kB 00:00 2026-02-13T10:36:34.464 INFO:teuthology.orchestra.run.trial121.stdout:(9/10): python3-rsa-4.9-2.el9.noarch.rpm 246 kB/s | 59 kB 00:00 2026-02-13T10:36:34.520 INFO:teuthology.orchestra.run.trial121.stdout:(10/10): python3-kubernetes-26.1.0-3.el9.noarch 3.2 MB/s | 1.0 MB 00:00 2026-02-13T10:36:34.523 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:34.523 INFO:teuthology.orchestra.run.trial121.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2026-02-13T10:36:34.548 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:34.556 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:34.556 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:34.595 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:34.596 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:34.668 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:34.718 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-13T10:36:34.730 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-13T10:36:34.745 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-13T10:36:34.749 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-13T10:36:34.757 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-13T10:36:34.795 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-13T10:36:34.807 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T10:36:35.120 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-13T10:36:35.134 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-13T10:36:35.135 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 10/10 2026-02-13T10:36:35.239 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 10/10 2026-02-13T10:36:35.239 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 1/10 2026-02-13T10:36:35.239 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-13T10:36:35.239 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-13T10:36:35.239 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-13T10:36:35.240 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-13T10:36:35.240 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-13T10:36:35.240 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T10:36:35.240 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-13T10:36:35.240 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-13T10:36:35.295 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-13T10:36:35.295 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T10:36:35.296 INFO:teuthology.orchestra.run.trial121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T10:36:35.297 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:35.297 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:35.327 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-mgr-cephadm 2026-02-13T10:36:35.632 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:38 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:35.716 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:35.717 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:35.717 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repository Size 2026-02-13T10:36:35.717 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:35.717 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:35.717 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-cephadm noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 150 k 2026-02-13T10:36:35.717 INFO:teuthology.orchestra.run.trial121.stdout:Installing dependencies: 2026-02-13T10:36:35.717 INFO:teuthology.orchestra.run.trial121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout:Install 6 Packages 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 7.0 M 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 33 M 2026-02-13T10:36:35.718 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:36.670 INFO:teuthology.orchestra.run.trial121.stdout:(1/6): ceph-mgr-cephadm-19.2.3-672.g283431c0.el 1.8 MB/s | 150 kB 00:00 2026-02-13T10:36:36.752 INFO:teuthology.orchestra.run.trial121.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.5 MB/s | 548 kB 00:00 2026-02-13T10:36:36.763 INFO:teuthology.orchestra.run.trial121.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 5.1 MB/s | 58 kB 00:00 2026-02-13T10:36:36.774 INFO:teuthology.orchestra.run.trial121.stdout:(4/6): python3-typing-extensions-4.15.0-1.el9.n 7.7 MB/s | 86 kB 00:00 2026-02-13T10:36:37.322 INFO:teuthology.orchestra.run.trial121.stdout:(5/6): python3-jinja2-2.11.3-8.el9.noarch.rpm 338 kB/s | 249 kB 00:00 2026-02-13T10:36:37.564 INFO:teuthology.orchestra.run.trial121.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 6.1 MB/s | 6.0 MB 00:00 2026-02-13T10:36:37.566 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:37.566 INFO:teuthology.orchestra.run.trial121.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2026-02-13T10:36:37.596 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:37.601 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:37.601 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:37.646 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:37.647 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:37.710 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:37.758 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/6 2026-02-13T10:36:37.768 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2026-02-13T10:36:37.986 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2026-02-13T10:36:38.009 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2026-02-13T10:36:38.030 INFO:teuthology.orchestra.run.trial121.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 5/6 2026-02-13T10:36:38.031 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 6/6 2026-02-13T10:36:38.140 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 6/6 2026-02-13T10:36:38.140 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/6 2026-02-13T10:36:38.140 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2026-02-13T10:36:38.140 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/6 2026-02-13T10:36:38.140 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2026-02-13T10:36:38.140 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2026-02-13T10:36:38.195 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 6/6 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:38.196 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:38.228 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install ceph-fuse 2026-02-13T10:36:38.527 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:41 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:38.610 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:38.611 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:38.611 INFO:teuthology.orchestra.run.trial121.stdout: Package Architecture Version Repository Size 2026-02-13T10:36:38.611 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:38.611 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout: ceph-fuse x86_64 2:19.2.3-672.g283431c0.el9 ceph 1.2 M 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 1.2 M 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 3.6 M 2026-02-13T10:36:38.612 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:38.742 INFO:teuthology.orchestra.run.trial121.stdout:ceph-fuse-19.2.3-672.g283431c0.el9.x86_64.rpm 8.8 MB/s | 1.2 MB 00:00 2026-02-13T10:36:38.742 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:38.743 INFO:teuthology.orchestra.run.trial121.stdout:Total 8.8 MB/s | 1.2 MB 00:00 2026-02-13T10:36:38.743 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:38.746 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:38.747 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:38.782 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:38.782 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:38.882 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:38.898 INFO:teuthology.orchestra.run.trial121.stdout: Installing : ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:39.118 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:39.173 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:39.173 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:39.173 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:39.173 INFO:teuthology.orchestra.run.trial121.stdout: ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:39.173 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:39.173 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:39.201 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install librados-devel 2026-02-13T10:36:39.507 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:42 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:39.591 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:39.592 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:39.592 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repo Size 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout: librados-devel x86_64 2:19.2.3-672.g283431c0.el9 ceph 126 k 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 126 k 2026-02-13T10:36:39.593 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 456 k 2026-02-13T10:36:39.594 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:39.678 INFO:teuthology.orchestra.run.trial121.stdout:librados-devel-19.2.3-672.g283431c0.el9.x86_64. 1.4 MB/s | 126 kB 00:00 2026-02-13T10:36:39.679 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:39.679 INFO:teuthology.orchestra.run.trial121.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-13T10:36:39.679 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:39.681 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:39.681 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:39.713 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:39.713 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:39.766 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:39.783 INFO:teuthology.orchestra.run.trial121.stdout: Installing : librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:39.867 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:39.924 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:39.924 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:39.924 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:39.925 INFO:teuthology.orchestra.run.trial121.stdout: librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:39.925 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:39.925 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:39.953 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install libcephfs2 2026-02-13T10:36:40.263 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:43 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:40.329 INFO:teuthology.orchestra.run.trial121.stdout:Package libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:40.347 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:40.347 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:40.347 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:40.370 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install libcephfs-devel 2026-02-13T10:36:40.672 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:43 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:40.757 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:40.757 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repo Size 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout: libcephfs-devel x86_64 2:19.2.3-672.g283431c0.el9 ceph 34 k 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:40.758 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:40.759 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 34 k 2026-02-13T10:36:40.759 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 153 k 2026-02-13T10:36:40.759 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:40.821 INFO:teuthology.orchestra.run.trial121.stdout:libcephfs-devel-19.2.3-672.g283431c0.el9.x86_64 535 kB/s | 34 kB 00:00 2026-02-13T10:36:40.821 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:40.822 INFO:teuthology.orchestra.run.trial121.stdout:Total 527 kB/s | 34 kB 00:00 2026-02-13T10:36:40.822 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:40.823 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:40.823 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:40.825 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:40.825 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:40.845 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:40.861 INFO:teuthology.orchestra.run.trial121.stdout: Installing : libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:40.919 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:40.975 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:40.975 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:40.976 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:40.976 INFO:teuthology.orchestra.run.trial121.stdout: libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:40.976 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:40.976 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:41.002 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install librados2 2026-02-13T10:36:41.299 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:44 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:41.365 INFO:teuthology.orchestra.run.trial121.stdout:Package librados2-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:41.383 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:41.383 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:41.384 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:41.405 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install librbd1 2026-02-13T10:36:41.703 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:44 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:41.769 INFO:teuthology.orchestra.run.trial121.stdout:Package librbd1-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:41.787 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:41.788 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:41.788 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:41.809 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install python3-rados 2026-02-13T10:36:42.114 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:45 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:42.181 INFO:teuthology.orchestra.run.trial121.stdout:Package python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:42.196 INFO:teuthology.orchestra.run.trial163.stdout:(8/10): python3-google-auth-2.45.0-1.el9.noarch 25 kB/s | 254 kB 00:10 2026-02-13T10:36:42.199 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:42.200 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:42.200 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:42.210 INFO:teuthology.orchestra.run.trial163.stdout:(9/10): python3-websocket-client-1.2.3-2.el9.no 8.9 kB/s | 90 kB 00:10 2026-02-13T10:36:42.221 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install python3-rgw 2026-02-13T10:36:42.237 INFO:teuthology.orchestra.run.trial163.stdout:(10/10): python3-kubernetes-26.1.0-3.el9.noarch 103 kB/s | 1.0 MB 00:10 2026-02-13T10:36:42.238 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:42.238 INFO:teuthology.orchestra.run.trial163.stdout:Total 189 kB/s | 2.0 MB 00:10 2026-02-13T10:36:42.263 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:42.270 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:42.270 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:42.309 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:42.309 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:42.384 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:42.435 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-13T10:36:42.448 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-13T10:36:42.463 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-13T10:36:42.467 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-13T10:36:42.475 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-13T10:36:42.514 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-13T10:36:42.527 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T10:36:42.541 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:45 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:42.594 INFO:teuthology.orchestra.run.trial121.stdout:Package python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:42.613 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:42.613 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:42.613 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:42.634 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install python3-cephfs 2026-02-13T10:36:42.846 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-13T10:36:42.861 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-13T10:36:42.861 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 10/10 2026-02-13T10:36:42.939 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:45 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 10/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 1/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-13T10:36:42.967 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-13T10:36:43.005 INFO:teuthology.orchestra.run.trial121.stdout:Package python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-rook-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T10:36:43.022 INFO:teuthology.orchestra.run.trial163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T10:36:43.023 INFO:teuthology.orchestra.run.trial163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T10:36:43.023 INFO:teuthology.orchestra.run.trial163.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T10:36:43.023 INFO:teuthology.orchestra.run.trial163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T10:36:43.023 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:43.023 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:43.023 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:43.024 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:43.024 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:43.046 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install python3-rbd 2026-02-13T10:36:43.053 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-mgr-cephadm 2026-02-13T10:36:43.344 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:46 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:43.358 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:43 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:43.409 INFO:teuthology.orchestra.run.trial121.stdout:Package python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:43.427 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:43.427 INFO:teuthology.orchestra.run.trial121.stdout:Nothing to do. 2026-02-13T10:36:43.428 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:43.441 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:43.441 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repository Size 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-cephadm noarch 2:19.2.3-672.g283431c0.el9 ceph-noarch 150 k 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout:Installing dependencies: 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T10:36:43.442 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:43.443 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:43.443 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:43.443 INFO:teuthology.orchestra.run.trial163.stdout:Install 6 Packages 2026-02-13T10:36:43.443 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:43.443 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 7.0 M 2026-02-13T10:36:43.444 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 33 M 2026-02-13T10:36:43.444 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:43.448 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install rbd-fuse 2026-02-13T10:36:43.700 INFO:teuthology.orchestra.run.trial163.stdout:(1/6): ceph-mgr-cephadm-19.2.3-672.g283431c0.el 1.6 MB/s | 150 kB 00:00 2026-02-13T10:36:43.749 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:46 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:43.836 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:43.837 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:43.837 INFO:teuthology.orchestra.run.trial121.stdout: Package Architecture Version Repository Size 2026-02-13T10:36:43.837 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout: rbd-fuse x86_64 2:19.2.3-672.g283431c0.el9 ceph 85 k 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 85 k 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 231 k 2026-02-13T10:36:43.838 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:43.914 INFO:teuthology.orchestra.run.trial121.stdout:rbd-fuse-19.2.3-672.g283431c0.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-13T10:36:43.915 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:43.915 INFO:teuthology.orchestra.run.trial121.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-13T10:36:43.915 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:43.918 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:43.919 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:43.949 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:43.950 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:44.000 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:44.016 INFO:teuthology.orchestra.run.trial121.stdout: Installing : rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:44.029 INFO:teuthology.orchestra.run.trial163.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.6 MB/s | 548 kB 00:00 2026-02-13T10:36:44.069 INFO:teuthology.orchestra.run.trial163.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.4 MB/s | 58 kB 00:00 2026-02-13T10:36:44.091 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:44.110 INFO:teuthology.orchestra.run.trial163.stdout:(4/6): python3-typing-extensions-4.15.0-1.el9.n 2.1 MB/s | 86 kB 00:00 2026-02-13T10:36:44.148 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:44.148 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:44.148 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:44.148 INFO:teuthology.orchestra.run.trial121.stdout: rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:44.148 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:44.148 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:44.176 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install rbd-mirror 2026-02-13T10:36:44.250 INFO:teuthology.orchestra.run.trial163.stdout:(5/6): python3-jinja2-2.11.3-8.el9.noarch.rpm 387 kB/s | 249 kB 00:00 2026-02-13T10:36:44.477 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:47 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:44.561 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:44.562 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:44.562 INFO:teuthology.orchestra.run.trial121.stdout: Package Arch Version Repo Size 2026-02-13T10:36:44.562 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout: rbd-mirror x86_64 2:19.2.3-672.g283431c0.el9 ceph 3.1 M 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 3.1 M 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 13 M 2026-02-13T10:36:44.563 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:44.566 INFO:teuthology.orchestra.run.trial163.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 6.2 MB/s | 6.0 MB 00:00 2026-02-13T10:36:44.568 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:44.569 INFO:teuthology.orchestra.run.trial163.stdout:Total 6.2 MB/s | 7.0 MB 00:01 2026-02-13T10:36:44.600 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:44.605 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:44.606 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:44.651 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:44.652 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:44.716 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:44.720 INFO:teuthology.orchestra.run.trial121.stdout:rbd-mirror-19.2.3-672.g283431c0.el9.x86_64.rpm 20 MB/s | 3.1 MB 00:00 2026-02-13T10:36:44.721 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:44.721 INFO:teuthology.orchestra.run.trial121.stdout:Total 20 MB/s | 3.1 MB 00:00 2026-02-13T10:36:44.721 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:44.725 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:44.725 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:44.765 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/6 2026-02-13T10:36:44.765 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:44.765 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:44.774 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2026-02-13T10:36:44.959 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:44.961 INFO:teuthology.orchestra.run.trial121.stdout: Installing : rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:44.984 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:44.984 INFO:teuthology.orchestra.run.trial121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:44.984 INFO:teuthology.orchestra.run.trial121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T10:36:44.984 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T10:36:44.985 INFO:teuthology.orchestra.run.trial121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T10:36:44.985 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:44.991 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2026-02-13T10:36:45.014 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2026-02-13T10:36:45.035 INFO:teuthology.orchestra.run.trial163.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 5/6 2026-02-13T10:36:45.036 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 6/6 2026-02-13T10:36:45.142 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 6/6 2026-02-13T10:36:45.142 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 1/6 2026-02-13T10:36:45.142 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2026-02-13T10:36:45.142 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/6 2026-02-13T10:36:45.142 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2026-02-13T10:36:45.142 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2026-02-13T10:36:45.197 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 6/6 2026-02-13T10:36:45.197 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:45.197 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:45.197 INFO:teuthology.orchestra.run.trial163.stdout: ceph-mgr-cephadm-2:19.2.3-672.g283431c0.el9.noarch 2026-02-13T10:36:45.198 INFO:teuthology.orchestra.run.trial163.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T10:36:45.198 INFO:teuthology.orchestra.run.trial163.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T10:36:45.198 INFO:teuthology.orchestra.run.trial163.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T10:36:45.198 INFO:teuthology.orchestra.run.trial163.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T10:36:45.198 INFO:teuthology.orchestra.run.trial163.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T10:36:45.198 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:45.198 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:45.228 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install ceph-fuse 2026-02-13T10:36:45.299 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:45.299 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:45.299 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:45.299 INFO:teuthology.orchestra.run.trial121.stdout: rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:45.300 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:45.300 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:45.328 DEBUG:teuthology.orchestra.run.trial121:> sudo yum -y install rbd-nbd 2026-02-13T10:36:45.524 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:45 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:45.606 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:45.607 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:45.607 INFO:teuthology.orchestra.run.trial163.stdout: Package Architecture Version Repository Size 2026-02-13T10:36:45.607 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:45.607 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout: ceph-fuse x86_64 2:19.2.3-672.g283431c0.el9 ceph 1.2 M 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 1.2 M 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 3.6 M 2026-02-13T10:36:45.608 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:45.633 INFO:teuthology.orchestra.run.trial121.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Feb 2026 10:35:57 AM UTC. 2026-02-13T10:36:45.717 INFO:teuthology.orchestra.run.trial121.stdout:Dependencies resolved. 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout: Package Architecture Version Repository Size 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:Installing: 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout: rbd-nbd x86_64 2:19.2.3-672.g283431c0.el9 ceph 171 k 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:Transaction Summary 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:================================================================================ 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:Install 1 Package 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:Total download size: 171 k 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:Installed size: 494 k 2026-02-13T10:36:45.718 INFO:teuthology.orchestra.run.trial121.stdout:Downloading Packages: 2026-02-13T10:36:45.740 INFO:teuthology.orchestra.run.trial163.stdout:ceph-fuse-19.2.3-672.g283431c0.el9.x86_64.rpm 8.6 MB/s | 1.2 MB 00:00 2026-02-13T10:36:45.741 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:45.741 INFO:teuthology.orchestra.run.trial163.stdout:Total 8.6 MB/s | 1.2 MB 00:00 2026-02-13T10:36:45.741 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:45.745 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:45.745 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:45.781 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:45.781 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:45.804 INFO:teuthology.orchestra.run.trial121.stdout:rbd-nbd-19.2.3-672.g283431c0.el9.x86_64.rpm 1.9 MB/s | 171 kB 00:00 2026-02-13T10:36:45.805 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:45.806 INFO:teuthology.orchestra.run.trial121.stdout:Total 1.9 MB/s | 171 kB 00:00 2026-02-13T10:36:45.806 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction check 2026-02-13T10:36:45.809 INFO:teuthology.orchestra.run.trial121.stdout:Transaction check succeeded. 2026-02-13T10:36:45.809 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction test 2026-02-13T10:36:45.840 INFO:teuthology.orchestra.run.trial121.stdout:Transaction test succeeded. 2026-02-13T10:36:45.840 INFO:teuthology.orchestra.run.trial121.stdout:Running transaction 2026-02-13T10:36:45.881 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:45.895 INFO:teuthology.orchestra.run.trial121.stdout: Preparing : 1/1 2026-02-13T10:36:45.897 INFO:teuthology.orchestra.run.trial163.stdout: Installing : ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:45.913 INFO:teuthology.orchestra.run.trial121.stdout: Installing : rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:45.991 INFO:teuthology.orchestra.run.trial121.stdout: Running scriptlet: rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:46.047 INFO:teuthology.orchestra.run.trial121.stdout: Verifying : rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:46.047 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:46.047 INFO:teuthology.orchestra.run.trial121.stdout:Installed: 2026-02-13T10:36:46.047 INFO:teuthology.orchestra.run.trial121.stdout: rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:46.047 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:36:46.047 INFO:teuthology.orchestra.run.trial121.stdout:Complete! 2026-02-13T10:36:46.076 DEBUG:teuthology.parallel:result is None 2026-02-13T10:36:46.131 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:46.186 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:46.186 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:46.186 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:46.187 INFO:teuthology.orchestra.run.trial163.stdout: ceph-fuse-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:46.187 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:46.187 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:46.213 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install librados-devel 2026-02-13T10:36:46.513 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:46 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:46.596 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repo Size 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout: librados-devel x86_64 2:19.2.3-672.g283431c0.el9 ceph 126 k 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 126 k 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 456 k 2026-02-13T10:36:46.597 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:46.684 INFO:teuthology.orchestra.run.trial163.stdout:librados-devel-19.2.3-672.g283431c0.el9.x86_64. 1.4 MB/s | 126 kB 00:00 2026-02-13T10:36:46.684 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:46.684 INFO:teuthology.orchestra.run.trial163.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-13T10:36:46.684 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:46.687 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:46.687 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:46.718 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:46.718 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:46.772 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:46.788 INFO:teuthology.orchestra.run.trial163.stdout: Installing : librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:46.862 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:46.917 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:46.918 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:46.918 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:46.918 INFO:teuthology.orchestra.run.trial163.stdout: librados-devel-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:46.918 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:46.918 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:46.946 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install libcephfs2 2026-02-13T10:36:47.243 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:47 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:47.308 INFO:teuthology.orchestra.run.trial163.stdout:Package libcephfs2-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:47.326 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:47.326 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:47.327 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:47.348 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install libcephfs-devel 2026-02-13T10:36:47.644 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:47 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:47.727 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repo Size 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout: libcephfs-devel x86_64 2:19.2.3-672.g283431c0.el9 ceph 34 k 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:47.728 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:47.729 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:47.729 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:47.729 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 34 k 2026-02-13T10:36:47.729 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 153 k 2026-02-13T10:36:47.729 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:47.790 INFO:teuthology.orchestra.run.trial163.stdout:libcephfs-devel-19.2.3-672.g283431c0.el9.x86_64 542 kB/s | 34 kB 00:00 2026-02-13T10:36:47.791 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:47.791 INFO:teuthology.orchestra.run.trial163.stdout:Total 534 kB/s | 34 kB 00:00 2026-02-13T10:36:47.791 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:47.793 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:47.793 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:47.795 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:47.795 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:47.817 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:47.833 INFO:teuthology.orchestra.run.trial163.stdout: Installing : libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:47.887 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:47.944 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:47.944 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:47.944 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:47.944 INFO:teuthology.orchestra.run.trial163.stdout: libcephfs-devel-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:47.945 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:47.945 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:47.971 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install librados2 2026-02-13T10:36:48.271 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:48.336 INFO:teuthology.orchestra.run.trial163.stdout:Package librados2-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:48.354 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:48.355 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:48.355 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:48.376 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install librbd1 2026-02-13T10:36:48.676 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:48 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:48.741 INFO:teuthology.orchestra.run.trial163.stdout:Package librbd1-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:48.759 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:48.759 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:48.760 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:48.781 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install python3-rados 2026-02-13T10:36:49.089 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:49.154 INFO:teuthology.orchestra.run.trial163.stdout:Package python3-rados-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:49.173 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:49.173 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:49.173 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:49.195 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install python3-rgw 2026-02-13T10:36:49.495 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:49.560 INFO:teuthology.orchestra.run.trial163.stdout:Package python3-rgw-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:49.578 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:49.579 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:49.579 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:49.600 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install python3-cephfs 2026-02-13T10:36:49.898 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:49.963 INFO:teuthology.orchestra.run.trial163.stdout:Package python3-cephfs-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:49.981 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:49.982 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:49.982 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:50.004 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install python3-rbd 2026-02-13T10:36:50.309 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:50.375 INFO:teuthology.orchestra.run.trial163.stdout:Package python3-rbd-2:19.2.3-672.g283431c0.el9.x86_64 is already installed. 2026-02-13T10:36:50.394 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:50.394 INFO:teuthology.orchestra.run.trial163.stdout:Nothing to do. 2026-02-13T10:36:50.395 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:50.417 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install rbd-fuse 2026-02-13T10:36:50.716 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:50.799 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:50.799 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:50.799 INFO:teuthology.orchestra.run.trial163.stdout: Package Architecture Version Repository Size 2026-02-13T10:36:50.799 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:50.799 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:50.799 INFO:teuthology.orchestra.run.trial163.stdout: rbd-fuse x86_64 2:19.2.3-672.g283431c0.el9 ceph 85 k 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 85 k 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 231 k 2026-02-13T10:36:50.800 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:50.873 INFO:teuthology.orchestra.run.trial163.stdout:rbd-fuse-19.2.3-672.g283431c0.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-13T10:36:50.874 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:50.874 INFO:teuthology.orchestra.run.trial163.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-13T10:36:50.874 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:50.877 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:50.877 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:50.908 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:50.908 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:50.963 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:50.980 INFO:teuthology.orchestra.run.trial163.stdout: Installing : rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:51.059 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:51.115 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:51.115 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:51.116 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:51.116 INFO:teuthology.orchestra.run.trial163.stdout: rbd-fuse-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:51.116 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:51.116 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:51.149 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install rbd-mirror 2026-02-13T10:36:51.446 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:51 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout: Package Arch Version Repo Size 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout: rbd-mirror x86_64 2:19.2.3-672.g283431c0.el9 ceph 3.1 M 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 3.1 M 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 13 M 2026-02-13T10:36:51.530 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:51.681 INFO:teuthology.orchestra.run.trial163.stdout:rbd-mirror-19.2.3-672.g283431c0.el9.x86_64.rpm 21 MB/s | 3.1 MB 00:00 2026-02-13T10:36:51.682 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:51.682 INFO:teuthology.orchestra.run.trial163.stdout:Total 21 MB/s | 3.1 MB 00:00 2026-02-13T10:36:51.682 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:51.685 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:51.686 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:51.725 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:51.726 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:51.921 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:51.922 INFO:teuthology.orchestra.run.trial163.stdout: Installing : rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:51.945 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:51.945 INFO:teuthology.orchestra.run.trial163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T10:36:51.945 INFO:teuthology.orchestra.run.trial163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T10:36:51.945 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T10:36:51.945 INFO:teuthology.orchestra.run.trial163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T10:36:51.946 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:52.272 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:52.272 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:52.272 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:52.273 INFO:teuthology.orchestra.run.trial163.stdout: rbd-mirror-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:52.273 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:52.273 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:52.301 DEBUG:teuthology.orchestra.run.trial163:> sudo yum -y install rbd-nbd 2026-02-13T10:36:52.600 INFO:teuthology.orchestra.run.trial163.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Feb 2026 10:36:00 AM UTC. 2026-02-13T10:36:52.683 INFO:teuthology.orchestra.run.trial163.stdout:Dependencies resolved. 2026-02-13T10:36:52.684 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:52.684 INFO:teuthology.orchestra.run.trial163.stdout: Package Architecture Version Repository Size 2026-02-13T10:36:52.684 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:52.684 INFO:teuthology.orchestra.run.trial163.stdout:Installing: 2026-02-13T10:36:52.684 INFO:teuthology.orchestra.run.trial163.stdout: rbd-nbd x86_64 2:19.2.3-672.g283431c0.el9 ceph 171 k 2026-02-13T10:36:52.684 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:52.685 INFO:teuthology.orchestra.run.trial163.stdout:Transaction Summary 2026-02-13T10:36:52.685 INFO:teuthology.orchestra.run.trial163.stdout:================================================================================ 2026-02-13T10:36:52.685 INFO:teuthology.orchestra.run.trial163.stdout:Install 1 Package 2026-02-13T10:36:52.685 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:52.685 INFO:teuthology.orchestra.run.trial163.stdout:Total download size: 171 k 2026-02-13T10:36:52.685 INFO:teuthology.orchestra.run.trial163.stdout:Installed size: 494 k 2026-02-13T10:36:52.685 INFO:teuthology.orchestra.run.trial163.stdout:Downloading Packages: 2026-02-13T10:36:52.769 INFO:teuthology.orchestra.run.trial163.stdout:rbd-nbd-19.2.3-672.g283431c0.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2026-02-13T10:36:52.776 INFO:teuthology.orchestra.run.trial163.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:36:52.776 INFO:teuthology.orchestra.run.trial163.stdout:Total 2.0 MB/s | 171 kB 00:00 2026-02-13T10:36:52.776 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction check 2026-02-13T10:36:52.777 INFO:teuthology.orchestra.run.trial163.stdout:Transaction check succeeded. 2026-02-13T10:36:52.777 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction test 2026-02-13T10:36:52.804 INFO:teuthology.orchestra.run.trial163.stdout:Transaction test succeeded. 2026-02-13T10:36:52.804 INFO:teuthology.orchestra.run.trial163.stdout:Running transaction 2026-02-13T10:36:52.859 INFO:teuthology.orchestra.run.trial163.stdout: Preparing : 1/1 2026-02-13T10:36:52.876 INFO:teuthology.orchestra.run.trial163.stdout: Installing : rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:52.963 INFO:teuthology.orchestra.run.trial163.stdout: Running scriptlet: rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:53.019 INFO:teuthology.orchestra.run.trial163.stdout: Verifying : rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 1/1 2026-02-13T10:36:53.019 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:53.019 INFO:teuthology.orchestra.run.trial163.stdout:Installed: 2026-02-13T10:36:53.019 INFO:teuthology.orchestra.run.trial163.stdout: rbd-nbd-2:19.2.3-672.g283431c0.el9.x86_64 2026-02-13T10:36:53.019 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:36:53.019 INFO:teuthology.orchestra.run.trial163.stdout:Complete! 2026-02-13T10:36:53.049 DEBUG:teuthology.parallel:result is None 2026-02-13T10:36:53.049 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T10:36:53.049 INFO:teuthology.packaging:ref: None 2026-02-13T10:36:53.049 INFO:teuthology.packaging:tag: None 2026-02-13T10:36:53.049 INFO:teuthology.packaging:branch: squid 2026-02-13T10:36:53.049 INFO:teuthology.packaging:sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:36:53.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2026-02-13T10:36:53.205 DEBUG:teuthology.orchestra.run.trial121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T10:36:53.225 INFO:teuthology.orchestra.run.trial121.stdout:19.2.3-672.g283431c0.el9 2026-02-13T10:36:53.225 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-672.g283431c0.el9 2026-02-13T10:36:53.225 INFO:teuthology.task.install:The correct ceph version 19.2.3-672.g283431c0 is installed. 2026-02-13T10:36:53.226 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T10:36:53.226 INFO:teuthology.packaging:ref: None 2026-02-13T10:36:53.226 INFO:teuthology.packaging:tag: None 2026-02-13T10:36:53.227 INFO:teuthology.packaging:branch: squid 2026-02-13T10:36:53.227 INFO:teuthology.packaging:sha1: 79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:36:53.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2026-02-13T10:36:53.372 DEBUG:teuthology.orchestra.run.trial163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T10:36:53.393 INFO:teuthology.orchestra.run.trial163.stdout:19.2.3-672.g283431c0.el9 2026-02-13T10:36:53.393 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-672.g283431c0.el9 2026-02-13T10:36:53.393 INFO:teuthology.task.install:The correct ceph version 19.2.3-672.g283431c0 is installed. 2026-02-13T10:36:53.395 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-13T10:36:53.396 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:36:53.396 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T10:36:53.422 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:36:53.422 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T10:36:53.458 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-13T10:36:53.458 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:36:53.459 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T10:36:53.488 DEBUG:teuthology.orchestra.run.trial121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T10:36:53.552 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:36:53.552 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T10:36:53.578 DEBUG:teuthology.orchestra.run.trial163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T10:36:53.642 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-13T10:36:53.642 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:36:53.642 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T10:36:53.668 DEBUG:teuthology.orchestra.run.trial121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T10:36:53.731 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:36:53.732 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T10:36:53.757 DEBUG:teuthology.orchestra.run.trial163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T10:36:53.821 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-13T10:36:53.821 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:36:53.821 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T10:36:53.846 DEBUG:teuthology.orchestra.run.trial121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T10:36:53.909 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:36:53.909 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T10:36:53.934 DEBUG:teuthology.orchestra.run.trial163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T10:36:54.000 INFO:teuthology.run_tasks:Running task print... 2026-02-13T10:36:54.018 INFO:teuthology.task.print:**** done install task... 2026-02-13T10:36:54.018 INFO:teuthology.run_tasks:Running task cephadm... 2026-02-13T10:36:54.147 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'squid', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'global': {'mon pg warn min per osd': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}}, 'image': 'quay.ceph.io/ceph-ci/ceph:squid', 'roleless': True, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'is offline because no MDS', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'BLUESTORE_SLOW_OP_ALERT', 'slow operation indications in BlueStore', 'experiencing slow operations in BlueStore', 'MGR_MODULE_ERROR', 'OSD_DOWN', 'osd.* is down', 'OSD_DOWN', 'osd.*is down', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'osds down', 'osd.* is down', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'pg .* is stuck inactive', 'pg .* is .*degraded', 'pg .* is stuck peering'], 'sha1': '79a8b77b79a4b70f9c11cf81d24d449d40916c58'} 2026-02-13T10:36:54.147 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:squid 2026-02-13T10:36:54.149 INFO:tasks.cephadm:Cluster fsid is e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:36:54.149 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2026-02-13T10:36:54.149 INFO:tasks.cephadm:No mon roles; fabricating mons 2026-02-13T10:36:54.149 INFO:tasks.cephadm:Monitor IPs: {'mon.trial121': '10.20.193.121', 'mon.trial163': '10.20.193.163'} 2026-02-13T10:36:54.149 INFO:tasks.cephadm:Normalizing hostnames... 2026-02-13T10:36:54.149 DEBUG:teuthology.orchestra.run.trial121:> sudo hostname $(hostname -s) 2026-02-13T10:36:54.175 DEBUG:teuthology.orchestra.run.trial163:> sudo hostname $(hostname -s) 2026-02-13T10:36:54.201 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for squid 2026-02-13T10:36:54.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=79a8b77b79a4b70f9c11cf81d24d449d40916c58 2026-02-13T10:36:54.318 INFO:tasks.cephadm:builder_project result: [{'url': 'https://2.chacra.ceph.com/r/ceph/wip-jcollin-testing-20260212.151113-tentacle/79a8b77b79a4b70f9c11cf81d24d449d40916c58/centos/9/flavors/default/', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-jcollin-testing-20260212.151113-tentacle/79a8b77b79a4b70f9c11cf81d24d449d40916c58/centos/9/flavors/default/', 'ref': 'wip-jcollin-testing-20260212.151113-tentacle', 'sha1': '79a8b77b79a4b70f9c11cf81d24d449d40916c58', 'distro': 'centos', 'distro_version': '9', 'distro_codename': None, 'modified': '2026-02-13 02:12:57.233661', 'status': 'ready', 'flavor': 'default', 'project': 'ceph', 'archs': ['source', 'arm64', 'x86_64'], 'extra': {'version': '20.2.0-667-g79a8b77b', 'package_manager_version': '20.2.0-667.g79a8b77b', 'build_url': 'https://jenkins.ceph.com/job/ceph-dev-pipeline/2951/', 'root_build_cause': '', 'node_name': '10.20.192.26+soko16', 'job_name': 'ceph-dev-pipeline'}}] 2026-02-13T10:36:54.485 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 283431c091dafc302f5753f809d0bd1eddce0869 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=squid 2026-02-13T10:36:54.486 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/x86_64/flavors/default/cephadm 2026-02-13T10:36:54.487 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/x86_64/flavors/default/cephadm 2026-02-13T10:36:54.487 DEBUG:teuthology.orchestra.run.trial121:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-13T10:36:54.818 INFO:teuthology.orchestra.run.trial121.stdout:-rw-r--r--. 1 ubuntu ubuntu 788157 Feb 13 10:36 /home/ubuntu/cephtest/cephadm 2026-02-13T10:36:54.819 DEBUG:teuthology.orchestra.run.trial163:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/283431c091dafc302f5753f809d0bd1eddce0869/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-13T10:36:55.120 INFO:teuthology.orchestra.run.trial163.stdout:-rw-r--r--. 1 ubuntu ubuntu 788157 Feb 13 10:36 /home/ubuntu/cephtest/cephadm 2026-02-13T10:36:55.121 DEBUG:teuthology.orchestra.run.trial121:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-13T10:36:55.140 DEBUG:teuthology.orchestra.run.trial163:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-13T10:36:55.175 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:squid on all hosts... 2026-02-13T10:36:55.175 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid pull 2026-02-13T10:36:55.182 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid pull 2026-02-13T10:36:55.455 INFO:teuthology.orchestra.run.trial121.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2026-02-13T10:36:55.476 INFO:teuthology.orchestra.run.trial163.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2026-02-13T10:37:06.814 INFO:teuthology.orchestra.run.trial121.stdout:{ 2026-02-13T10:37:06.814 INFO:teuthology.orchestra.run.trial121.stdout: "ceph_version": "ceph version 19.2.3-672-g283431c0 (283431c091dafc302f5753f809d0bd1eddce0869) squid (stable)", 2026-02-13T10:37:06.814 INFO:teuthology.orchestra.run.trial121.stdout: "image_id": "4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126", 2026-02-13T10:37:06.814 INFO:teuthology.orchestra.run.trial121.stdout: "repo_digests": [ 2026-02-13T10:37:06.814 INFO:teuthology.orchestra.run.trial121.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4" 2026-02-13T10:37:06.814 INFO:teuthology.orchestra.run.trial121.stdout: ] 2026-02-13T10:37:06.814 INFO:teuthology.orchestra.run.trial121.stdout:} 2026-02-13T10:37:06.831 INFO:teuthology.orchestra.run.trial163.stdout:{ 2026-02-13T10:37:06.831 INFO:teuthology.orchestra.run.trial163.stdout: "ceph_version": "ceph version 19.2.3-672-g283431c0 (283431c091dafc302f5753f809d0bd1eddce0869) squid (stable)", 2026-02-13T10:37:06.832 INFO:teuthology.orchestra.run.trial163.stdout: "image_id": "4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126", 2026-02-13T10:37:06.832 INFO:teuthology.orchestra.run.trial163.stdout: "repo_digests": [ 2026-02-13T10:37:06.832 INFO:teuthology.orchestra.run.trial163.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4" 2026-02-13T10:37:06.832 INFO:teuthology.orchestra.run.trial163.stdout: ] 2026-02-13T10:37:06.832 INFO:teuthology.orchestra.run.trial163.stdout:} 2026-02-13T10:37:06.848 DEBUG:teuthology.orchestra.run.trial121:> sudo mkdir -p /etc/ceph 2026-02-13T10:37:06.892 DEBUG:teuthology.orchestra.run.trial163:> sudo mkdir -p /etc/ceph 2026-02-13T10:37:06.920 DEBUG:teuthology.orchestra.run.trial121:> sudo chmod 777 /etc/ceph 2026-02-13T10:37:06.959 DEBUG:teuthology.orchestra.run.trial163:> sudo chmod 777 /etc/ceph 2026-02-13T10:37:06.984 INFO:tasks.cephadm:Writing seed config... 2026-02-13T10:37:06.986 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2026-02-13T10:37:06.986 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] debug ms = 1 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] debug osd = 20 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2026-02-13T10:37:06.987 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [mon] debug mon = 20 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [mon] debug ms = 1 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2026-02-13T10:37:06.988 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2026-02-13T10:37:06.989 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:06.989 DEBUG:teuthology.orchestra.run.trial121:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2026-02-13T10:37:07.018 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e9ef2631-08c7-11f1-a2b3-d404e6e7d460 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd mclock skip benchmark = true osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2026-02-13T10:37:07.018 DEBUG:teuthology.orchestra.run.trial121:mon.trial121> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121.service 2026-02-13T10:37:07.060 INFO:tasks.cephadm:Bootstrapping... 2026-02-13T10:37:07.060 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid -v bootstrap --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 10.20.193.121 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:07.326 INFO:teuthology.orchestra.run.trial121.stdout:-------------------------------------------------------------------------------- 2026-02-13T10:37:07.326 INFO:teuthology.orchestra.run.trial121.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:squid', '-v', 'bootstrap', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '10.20.193.121', '--skip-admin-label'] 2026-02-13T10:37:07.327 INFO:teuthology.orchestra.run.trial121.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2026-02-13T10:37:07.327 INFO:teuthology.orchestra.run.trial121.stdout:Verifying podman|docker is present... 2026-02-13T10:37:07.342 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stdout 5.6.0 2026-02-13T10:37:07.342 INFO:teuthology.orchestra.run.trial121.stdout:Verifying lvm2 is present... 2026-02-13T10:37:07.342 INFO:teuthology.orchestra.run.trial121.stdout:Verifying time synchronization is in place... 2026-02-13T10:37:07.348 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-13T10:37:07.348 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-13T10:37:07.353 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-13T10:37:07.354 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout inactive 2026-02-13T10:37:07.360 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout enabled 2026-02-13T10:37:07.366 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout active 2026-02-13T10:37:07.366 INFO:teuthology.orchestra.run.trial121.stdout:Unit chronyd.service is enabled and running 2026-02-13T10:37:07.366 INFO:teuthology.orchestra.run.trial121.stdout:Repeating the final host check... 2026-02-13T10:37:07.381 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stdout 5.6.0 2026-02-13T10:37:07.382 INFO:teuthology.orchestra.run.trial121.stdout:podman (/bin/podman) version 5.6.0 is present 2026-02-13T10:37:07.382 INFO:teuthology.orchestra.run.trial121.stdout:systemctl is present 2026-02-13T10:37:07.382 INFO:teuthology.orchestra.run.trial121.stdout:lvcreate is present 2026-02-13T10:37:07.387 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-13T10:37:07.387 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-13T10:37:07.393 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-13T10:37:07.393 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout inactive 2026-02-13T10:37:07.399 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout enabled 2026-02-13T10:37:07.405 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout active 2026-02-13T10:37:07.406 INFO:teuthology.orchestra.run.trial121.stdout:Unit chronyd.service is enabled and running 2026-02-13T10:37:07.406 INFO:teuthology.orchestra.run.trial121.stdout:Host looks OK 2026-02-13T10:37:07.406 INFO:teuthology.orchestra.run.trial121.stdout:Cluster fsid: e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:07.406 INFO:teuthology.orchestra.run.trial121.stdout:Acquiring lock 139777679103264 on /run/cephadm/e9ef2631-08c7-11f1-a2b3-d404e6e7d460.lock 2026-02-13T10:37:07.406 INFO:teuthology.orchestra.run.trial121.stdout:Lock 139777679103264 acquired on /run/cephadm/e9ef2631-08c7-11f1-a2b3-d404e6e7d460.lock 2026-02-13T10:37:07.406 INFO:teuthology.orchestra.run.trial121.stdout:Verifying IP 10.20.193.121 port 3300 ... 2026-02-13T10:37:07.407 INFO:teuthology.orchestra.run.trial121.stdout:Verifying IP 10.20.193.121 port 6789 ... 2026-02-13T10:37:07.407 INFO:teuthology.orchestra.run.trial121.stdout:Base mon IP(s) is [10.20.193.121:3300, 10.20.193.121:6789], mon addrv is [v2:10.20.193.121:3300,v1:10.20.193.121:6789] 2026-02-13T10:37:07.409 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout default via 10.20.192.1 dev enp1s0f0 proto dhcp src 10.20.193.121 metric 102 2026-02-13T10:37:07.409 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout 10.20.192.0/20 dev enp1s0f0 proto kernel scope link src 10.20.193.121 metric 102 2026-02-13T10:37:07.409 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout 169.254.3.0/24 dev enp7s0f4u2u2c2 proto kernel scope link src 169.254.3.1 metric 101 2026-02-13T10:37:07.412 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2026-02-13T10:37:07.412 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout fe80::/64 dev enp1s0f0 proto kernel metric 256 pref medium 2026-02-13T10:37:07.412 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout fe80::/64 dev enp7s0f4u2u2c2 proto kernel metric 1024 pref medium 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout 2: enp7s0f4u2u2c2: mtu 1500 state UNKNOWN qlen 1000 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout inet6 fe80::ade3:66c0:f8e2:5ec4/64 scope link noprefixroute 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout 3: enp1s0f0: mtu 1500 state UP qlen 1000 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout inet6 fe80::925a:8ff:fe77:61fc/64 scope link 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-13T10:37:07.416 INFO:teuthology.orchestra.run.trial121.stdout:Mon IP `10.20.193.121` is in CIDR network `10.20.192.0/20` 2026-02-13T10:37:07.417 INFO:teuthology.orchestra.run.trial121.stdout:Mon IP `10.20.193.121` is in CIDR network `10.20.192.0/20` 2026-02-13T10:37:07.417 INFO:teuthology.orchestra.run.trial121.stdout:Inferred mon public CIDR from local network configuration ['10.20.192.0/20', '10.20.192.0/20'] 2026-02-13T10:37:07.417 INFO:teuthology.orchestra.run.trial121.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2026-02-13T10:37:07.417 INFO:teuthology.orchestra.run.trial121.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2026-02-13T10:37:07.645 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stdout 4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 2026-02-13T10:37:07.645 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:squid... 2026-02-13T10:37:07.645 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stderr Getting image source signatures 2026-02-13T10:37:07.646 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stderr Copying blob sha256:64e40020cdc81486fe697dfe0e523a5804d33e01a9e9bdfc58898ab18d415677 2026-02-13T10:37:07.646 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stderr Copying blob sha256:c92ccfe8c151fb35c12ff15a7a7de08028e120d73814d436eadc192e84f49ad9 2026-02-13T10:37:07.646 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stderr Copying config sha256:4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 2026-02-13T10:37:07.646 INFO:teuthology.orchestra.run.trial121.stdout:/bin/podman: stderr Writing manifest to image destination 2026-02-13T10:37:07.758 INFO:teuthology.orchestra.run.trial121.stdout:ceph: stdout ceph version 19.2.3-672-g283431c0 (283431c091dafc302f5753f809d0bd1eddce0869) squid (stable) 2026-02-13T10:37:07.758 INFO:teuthology.orchestra.run.trial121.stdout:Ceph version: ceph version 19.2.3-672-g283431c0 (283431c091dafc302f5753f809d0bd1eddce0869) squid (stable) 2026-02-13T10:37:07.758 INFO:teuthology.orchestra.run.trial121.stdout:Extracting ceph user uid/gid from container image... 2026-02-13T10:37:07.878 INFO:teuthology.orchestra.run.trial121.stdout:stat: stdout 167 167 2026-02-13T10:37:07.879 INFO:teuthology.orchestra.run.trial121.stdout:Creating initial keys... 2026-02-13T10:37:08.009 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph-authtool: stdout AQDT/o5pN/7JOhAAp/bIsEAHhs704PwoZPA1Hg== 2026-02-13T10:37:08.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph-authtool: stdout AQDU/o5phhGVBhAAhZOckQwiQfWVaSJVeIswqg== 2026-02-13T10:37:08.281 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph-authtool: stdout AQDU/o5pH3MNEBAAXyXK8p11tKOhWfVF8YhkQA== 2026-02-13T10:37:08.282 INFO:teuthology.orchestra.run.trial121.stdout:Creating initial monmap... 2026-02-13T10:37:08.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-13T10:37:08.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2026-02-13T10:37:08.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:08.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-13T10:37:08.399 INFO:teuthology.orchestra.run.trial121.stdout:monmaptool for trial121 [v2:10.20.193.121:3300,v1:10.20.193.121:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-13T10:37:08.399 INFO:teuthology.orchestra.run.trial121.stdout:setting min_mon_release = quincy 2026-02-13T10:37:08.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/monmaptool: set fsid to e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:08.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-13T10:37:08.399 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:08.399 INFO:teuthology.orchestra.run.trial121.stdout:Creating mon... 2026-02-13T10:37:08.522 INFO:teuthology.orchestra.run.trial121.stdout:create mon.trial121 on 2026-02-13T10:37:08.689 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2026-02-13T10:37:08.788 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2026-02-13T10:37:08.900 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460.target → /etc/systemd/system/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460.target. 2026-02-13T10:37:08.900 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460.target → /etc/systemd/system/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460.target. 2026-02-13T10:37:09.027 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121 2026-02-13T10:37:09.027 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121.service: Unit ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121.service not loaded. 2026-02-13T10:37:09.150 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460.target.wants/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121.service → /etc/systemd/system/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@.service. 2026-02-13T10:37:09.300 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-13T10:37:09.300 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout disabled 2026-02-13T10:37:09.307 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-13T10:37:09.307 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout inactive 2026-02-13T10:37:09.307 INFO:teuthology.orchestra.run.trial121.stdout:firewalld.service is not enabled 2026-02-13T10:37:09.307 INFO:teuthology.orchestra.run.trial121.stdout:Not possible to enable service . firewalld.service is not available 2026-02-13T10:37:09.307 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for mon to start... 2026-02-13T10:37:09.307 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for mon... 2026-02-13T10:37:09.464 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout cluster: 2026-02-13T10:37:09.464 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout id: e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:09.464 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2026-02-13T10:37:09.464 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:09.464 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout services: 2026-02-13T10:37:09.464 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum trial121 (age 0.133845s) 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout data: 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout pgs: 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:mon is available 2026-02-13T10:37:09.465 INFO:teuthology.orchestra.run.trial121.stdout:Assimilating anything we can from ceph.conf... 2026-02-13T10:37:09.616 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:09.616 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout [global] 2026-02-13T10:37:09.616 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout fsid = e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:09.616 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-13T10:37:09.616 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.121:3300,v1:10.20.193.121:6789] 2026-02-13T10:37:09.616 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-13T10:37:09.616 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout [osd] 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-13T10:37:09.617 INFO:teuthology.orchestra.run.trial121.stdout:Generating new minimal ceph.conf... 2026-02-13T10:37:09.775 INFO:teuthology.orchestra.run.trial121.stdout:Restarting the monitor... 2026-02-13T10:37:10.100 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:09 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121[18724]: 2026-02-13T10:37:09.846+0000 7f5d86448640 -1 mon.trial121@0(leader) e1 *** Got Signal Terminated *** 2026-02-13T10:37:10.100 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 podman[19008]: 2026-02-13 10:37:10.099704736 +0000 UTC m=+0.265673973 container died 2eda0fc22314fbaffa8a6af7bf88e08d5453432574738415f7536a641ba27662 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121, org.label-schema.schema-version=1.0, ceph=True, CEPH_REF=squid, org.label-schema.build-date=20260202, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.label-schema.license=GPLv2) 2026-02-13T10:37:10.289 INFO:teuthology.orchestra.run.trial121.stdout:Setting public_network to 10.20.192.0/20 in mon config section 2026-02-13T10:37:10.360 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 podman[19008]: 2026-02-13 10:37:10.107165019 +0000 UTC m=+0.273134266 container remove 2eda0fc22314fbaffa8a6af7bf88e08d5453432574738415f7536a641ba27662 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260202, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, CEPH_REF=squid, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, org.label-schema.vendor=CentOS) 2026-02-13T10:37:10.360 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 bash[19008]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121 2026-02-13T10:37:10.360 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 systemd[1]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121.service: Deactivated successfully. 2026-02-13T10:37:10.360 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 systemd[1]: Stopped Ceph mon.trial121 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460. 2026-02-13T10:37:10.360 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 systemd[1]: Starting Ceph mon.trial121 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:37:10.360 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 podman[19130]: 2026-02-13 10:37:10.257591404 +0000 UTC m=+0.017559565 container create c5ae73bd5b1b658d8fd3bacccf4773add98d7988290010d4274886efcb1b9887 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121, org.label-schema.build-date=20260202, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=squid, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T10:37:10.360 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 podman[19130]: 2026-02-13 10:37:10.281666027 +0000 UTC m=+0.041634118 container init c5ae73bd5b1b658d8fd3bacccf4773add98d7988290010d4274886efcb1b9887 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_REF=squid, org.label-schema.build-date=20260202, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, ceph=True, io.buildah.version=1.41.3, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 podman[19130]: 2026-02-13 10:37:10.284025235 +0000 UTC m=+0.043993316 container start c5ae73bd5b1b658d8fd3bacccf4773add98d7988290010d4274886efcb1b9887 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, CEPH_REF=squid, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, io.buildah.version=1.41.3, org.label-schema.build-date=20260202, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 bash[19130]: c5ae73bd5b1b658d8fd3bacccf4773add98d7988290010d4274886efcb1b9887 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 podman[19130]: 2026-02-13 10:37:10.249834189 +0000 UTC m=+0.009802280 image pull 4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 quay.ceph.io/ceph-ci/ceph:squid 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 systemd[1]: Started Ceph mon.trial121 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460. 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: set uid:gid to 167:167 (ceph:ceph) 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: ceph version 19.2.3-672-g283431c0 (283431c091dafc302f5753f809d0bd1eddce0869) squid (stable), process ceph-mon, pid 2 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: pidfile_write: ignore empty --pid-file 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: load: jerasure load: lrc 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: RocksDB version: 7.9.2 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Git sha 0 2026-02-13T10:37:10.361 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Compile date 2026-02-07 14:59:59 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: DB SUMMARY 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: DB Session ID: SENBDG6X4JZPJ8LGORYC 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: CURRENT file: CURRENT 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: IDENTITY file: IDENTITY 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: SST files in /var/lib/ceph/mon/ceph-trial121/store.db dir, Total Num: 1, files: 000008.sst 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-trial121/store.db: 000009.log size: 81263 ; 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.error_if_exists: 0 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.create_if_missing: 0 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.paranoid_checks: 1 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.flush_verify_memtable_count: 1 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-13T10:37:10.362 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.env: 0x55bc71da9dc0 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.fs: PosixFileSystem 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.info_log: 0x55bc72673a20 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_file_opening_threads: 16 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.statistics: (nil) 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.use_fsync: 0 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_log_file_size: 0 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.log_file_time_to_roll: 0 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.keep_log_file_num: 1000 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.recycle_log_file_num: 0 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.allow_fallocate: 1 2026-02-13T10:37:10.363 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.allow_mmap_reads: 0 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.allow_mmap_writes: 0 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.use_direct_reads: 0 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.create_missing_column_families: 0 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.db_log_dir: 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.wal_dir: 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.table_cache_numshardbits: 6 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.WAL_ttl_seconds: 0 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.WAL_size_limit_MB: 0 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-13T10:37:10.364 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.is_fd_close_on_exec: 1 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.advise_random_on_open: 1 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.db_write_buffer_size: 0 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.write_buffer_manager: 0x55bc72677900 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.use_adaptive_mutex: 0 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.rate_limiter: (nil) 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.wal_recovery_mode: 2 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.enable_thread_tracking: 0 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.enable_pipelined_write: 0 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.unordered_write: 0 2026-02-13T10:37:10.365 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.row_cache: None 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.wal_filter: None 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.allow_ingest_behind: 0 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.two_write_queues: 0 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.manual_wal_flush: 0 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.wal_compression: 0 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.atomic_flush: 0 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-13T10:37:10.366 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.persist_stats_to_disk: 0 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.write_dbid_to_manifest: 0 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.log_readahead_size: 0 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.best_efforts_recovery: 0 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.allow_data_in_errors: 0 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.db_host_id: __hostname__ 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.enforce_single_del_contracts: true 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_background_jobs: 2 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_background_compactions: -1 2026-02-13T10:37:10.367 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_subcompactions: 1 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.delayed_write_rate : 16777216 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_total_wal_size: 0 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.stats_dump_period_sec: 600 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.stats_persist_period_sec: 600 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_open_files: -1 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bytes_per_sync: 0 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.wal_bytes_per_sync: 0 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.strict_bytes_per_sync: 0 2026-02-13T10:37:10.368 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_readahead_size: 0 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_background_flushes: -1 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Compression algorithms supported: 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kZSTD supported: 0 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kXpressCompression supported: 0 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kBZip2Compression supported: 0 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kLZ4Compression supported: 1 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kZlibCompression supported: 1 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kLZ4HCCompression supported: 1 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: kSnappyCompression supported: 1 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Fast CRC32 supported: Supported on x86 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: DMutex implementation: pthread_mutex_t 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-trial121/store.db/MANIFEST-000010 2026-02-13T10:37:10.369 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.merge_operator: 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_filter: None 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_filter_factory: None 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.sst_partitioner_factory: None 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.memtable_factory: SkipListFactory 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.table_factory: BlockBasedTable 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bc72672700) 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout: cache_index_and_filter_blocks: 1 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2026-02-13T10:37:10.370 INFO:journalctl@ceph.mon.trial121.trial121.stdout: pin_top_level_index_and_filter: 1 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: index_type: 0 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: data_block_index_type: 0 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: index_shortening: 1 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: data_block_hash_table_util_ratio: 0.750000 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: checksum: 4 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: no_block_cache: 0 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_cache: 0x55bc72697350 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_cache_name: BinnedLRUCache 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_cache_options: 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: capacity : 536870912 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: num_shard_bits : 4 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: strict_capacity_limit : 0 2026-02-13T10:37:10.371 INFO:journalctl@ceph.mon.trial121.trial121.stdout: high_pri_pool_ratio: 0.000 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_cache_compressed: (nil) 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: persistent_cache: (nil) 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_size: 4096 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_size_deviation: 10 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_restart_interval: 16 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: index_block_restart_interval: 1 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: metadata_block_size: 4096 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: partition_filters: 0 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: use_delta_encoding: 1 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: filter_policy: bloomfilter 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: whole_key_filtering: 1 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: verify_compression: 0 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: read_amp_bytes_per_bit: 0 2026-02-13T10:37:10.372 INFO:journalctl@ceph.mon.trial121.trial121.stdout: format_version: 5 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout: enable_index_compression: 1 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout: block_align: 0 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout: max_auto_readahead_size: 262144 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout: prepopulate_block_cache: 0 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout: initial_auto_readahead_size: 8192 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout: num_file_reads_for_auto_readahead: 2 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.write_buffer_size: 33554432 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_write_buffer_number: 2 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression: NoCompression 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression: Disabled 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.prefix_extractor: nullptr 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-13T10:37:10.373 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.num_levels: 7 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.window_bits: -14 2026-02-13T10:37:10.374 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.level: 32767 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.strategy: 0 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.enabled: false 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.target_file_size_base: 67108864 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.target_file_size_multiplier: 1 2026-02-13T10:37:10.375 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_base: 268435456 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.arena_block_size: 1048576 2026-02-13T10:37:10.376 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.disable_auto_compactions: 0 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-13T10:37:10.377 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.inplace_update_support: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.inplace_update_num_locks: 10000 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.memtable_huge_page_size: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.bloom_locality: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.max_successive_merges: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.optimize_filters_for_hits: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.paranoid_file_checks: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.force_consistency_checks: 1 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.report_bg_io_stats: 0 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.ttl: 2592000 2026-02-13T10:37:10.378 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.periodic_compaction_seconds: 0 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.enable_blob_files: false 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.min_blob_size: 0 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.blob_file_size: 268435456 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.blob_compression_type: NoCompression 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.enable_blob_garbage_collection: false 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.blob_file_starting_level: 0 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-13T10:37:10.379 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-trial121/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fa4420e9-9778-4e46-8fb4-754482852a19 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: EVENT_LOG_v1 {"time_micros": 1770979030301099, "job": 1, "event": "recovery_started", "wal_files": [9]} 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: EVENT_LOG_v1 {"time_micros": 1770979030301821, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 77722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 251, "table_properties": {"data_size": 75915, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 11168, "raw_average_key_size": 50, "raw_value_size": 69769, "raw_average_value_size": 314, "num_data_blocks": 8, "num_entries": 222, "num_filter_entries": 222, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770979030, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fa4420e9-9778-4e46-8fb4-754482852a19", "db_session_id": "SENBDG6X4JZPJ8LGORYC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: EVENT_LOG_v1 {"time_micros": 1770979030301895, "job": 1, "event": "recovery_finished"} 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-trial121/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bc72698e00 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: DB pointer 0x55bc727a6000 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: rocksdb: [db/db_impl/db_impl.cc:1111] 2026-02-13T10:37:10.380 INFO:journalctl@ceph.mon.trial121.trial121.stdout: ** DB Stats ** 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: ** Compaction Stats [default] ** 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2026-02-13T10:37:10.381 INFO:journalctl@ceph.mon.trial121.trial121.stdout: L0 2/0 77.79 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Sum 2/0 77.79 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: ** Compaction Stats [default] ** 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 153.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-13T10:37:10.382 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Flush(GB): cumulative 0.000, interval 0.000 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: AddFile(Total Files): cumulative 0, interval 0 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: AddFile(L0 Files): cumulative 0, interval 0 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: AddFile(Keys): cumulative 0, interval 0 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Cumulative compaction: 0.00 GB write, 24.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Interval compaction: 0.00 GB write, 24.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Block cache BinnedLRUCache@0x55bc72697350#2 capacity: 512.00 MB usage: 1.14 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.38 KB,7.15256e-05%) Misc(1,0.00 KB,0%) 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout: ** File Read Latency Histogram By Level [default] ** 2026-02-13T10:37:10.383 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: starting mon.trial121 rank 0 at public addrs [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] at bind addrs [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] mon_data /var/lib/ceph/mon/ceph-trial121 fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???) e1 preinit fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???).mds e1 new map 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???).mds e1 print_map 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout: e1 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout: btime 2026-02-13T10:37:09:310939+0000 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout: enable_multiple, ever_enabled_multiple: 1,1 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout: legacy client fscid: -1 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout: 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout: No filesystems configured 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-13T10:37:10.384 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mon.trial121 is new leader, mons trial121 in quorum (ranks 0) 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: monmap epoch 1 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: last_changed 2026-02-13T10:37:08.375508+0000 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: created 2026-02-13T10:37:08.375508+0000 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: min_mon_release 19 (squid) 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: election_strategy: 1 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: 0: [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] mon.trial121 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: fsmap 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: osdmap e1: 0 total, 0 up, 0 in 2026-02-13T10:37:10.385 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:10 trial121 ceph-mon[19159]: mgrmap e1: no daemons active 2026-02-13T10:37:10.444 INFO:teuthology.orchestra.run.trial121.stdout:Wrote config to /etc/ceph/ceph.conf 2026-02-13T10:37:10.445 INFO:teuthology.orchestra.run.trial121.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:10.445 INFO:teuthology.orchestra.run.trial121.stdout:Creating mgr... 2026-02-13T10:37:10.445 INFO:teuthology.orchestra.run.trial121.stdout:Verifying port 0.0.0.0:9283 ... 2026-02-13T10:37:10.445 INFO:teuthology.orchestra.run.trial121.stdout:Verifying port 0.0.0.0:8765 ... 2026-02-13T10:37:10.446 INFO:teuthology.orchestra.run.trial121.stdout:Verifying port 0.0.0.0:8443 ... 2026-02-13T10:37:10.573 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mgr.trial121.tkoihk 2026-02-13T10:37:10.574 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mgr.trial121.tkoihk.service: Unit ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mgr.trial121.tkoihk.service not loaded. 2026-02-13T10:37:10.677 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460.target.wants/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mgr.trial121.tkoihk.service → /etc/systemd/system/ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@.service. 2026-02-13T10:37:10.816 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-13T10:37:10.816 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout disabled 2026-02-13T10:37:10.822 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-13T10:37:10.822 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout inactive 2026-02-13T10:37:10.822 INFO:teuthology.orchestra.run.trial121.stdout:firewalld.service is not enabled 2026-02-13T10:37:10.822 INFO:teuthology.orchestra.run.trial121.stdout:Not possible to enable service . firewalld.service is not available 2026-02-13T10:37:10.829 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-13T10:37:10.829 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout disabled 2026-02-13T10:37:10.835 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-13T10:37:10.836 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout inactive 2026-02-13T10:37:10.836 INFO:teuthology.orchestra.run.trial121.stdout:firewalld.service is not enabled 2026-02-13T10:37:10.836 INFO:teuthology.orchestra.run.trial121.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2026-02-13T10:37:10.836 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for mgr to start... 2026-02-13T10:37:10.836 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for mgr... 2026-02-13T10:37:11.002 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout { 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "fsid": "e9ef2631-08c7-11f1-a2b3-d404e6e7d460", 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "health": { 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-13T10:37:11.003 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 0 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "trial121" 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum_age": 0, 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-13T10:37:11.004 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-13T10:37:11.005 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "btime": "2026-02-13T10:37:09:310939+0000", 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "available": false, 2026-02-13T10:37:11.006 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "nfs", 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "restful" 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "modified": "2026-02-13T10:37:09.311279+0000", 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:11.007 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-13T10:37:11.008 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout } 2026-02-13T10:37:11.008 INFO:teuthology.orchestra.run.trial121.stdout:mgr not available, waiting (1/15)... 2026-02-13T10:37:11.621 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:11 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2910009093' entity='client.admin' 2026-02-13T10:37:11.621 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:11 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2816995506' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout { 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "fsid": "e9ef2631-08c7-11f1-a2b3-d404e6e7d460", 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "health": { 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 0 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "trial121" 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:13.155 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-13T10:37:13.156 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "btime": "2026-02-13T10:37:09:310939+0000", 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "available": false, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "nfs", 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "restful" 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "modified": "2026-02-13T10:37:09.311279+0000", 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout } 2026-02-13T10:37:13.157 INFO:teuthology.orchestra.run.trial121.stdout:mgr not available, waiting (2/15)... 2026-02-13T10:37:13.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:13 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2883629661' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2026-02-13T10:37:14.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: Activating manager daemon trial121.tkoihk 2026-02-13T10:37:14.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: mgrmap e2: trial121.tkoihk(active, starting, since 0.00258744s) 2026-02-13T10:37:14.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata"}]: dispatch 2026-02-13T10:37:14.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata"}]: dispatch 2026-02-13T10:37:14.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata"}]: dispatch 2026-02-13T10:37:14.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:37:14.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr metadata", "who": "trial121.tkoihk", "id": "trial121.tkoihk"}]: dispatch 2026-02-13T10:37:14.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: Manager daemon trial121.tkoihk is now available 2026-02-13T10:37:14.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/mirror_snapshot_schedule"}]: dispatch 2026-02-13T10:37:14.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/trash_purge_schedule"}]: dispatch 2026-02-13T10:37:14.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:14.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:14.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:14 trial121 ceph-mon[19159]: from='mgr.14100 10.20.193.121:0/578483515' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout { 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "fsid": "e9ef2631-08c7-11f1-a2b3-d404e6e7d460", 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "health": { 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-13T10:37:15.398 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 0 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "trial121" 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2026-02-13T10:37:15.399 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-13T10:37:15.400 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "btime": "2026-02-13T10:37:09:310939+0000", 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-13T10:37:15.401 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "nfs", 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "restful" 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ], 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T10:37:15.402 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "modified": "2026-02-13T10:37:09.311279+0000", 2026-02-13T10:37:15.403 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T10:37:15.403 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout }, 2026-02-13T10:37:15.403 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-13T10:37:15.403 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout } 2026-02-13T10:37:15.403 INFO:teuthology.orchestra.run.trial121.stdout:mgr is available 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout [global] 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout fsid = e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.121:3300,v1:10.20.193.121:6789] 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout [osd] 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-13T10:37:15.599 INFO:teuthology.orchestra.run.trial121.stdout:Enabling cephadm module... 2026-02-13T10:37:15.707 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:15 trial121 ceph-mon[19159]: mgrmap e3: trial121.tkoihk(active, since 1.00432s) 2026-02-13T10:37:15.707 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:15 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3339032746' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2026-02-13T10:37:16.727 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:16 trial121 ceph-mon[19159]: mgrmap e4: trial121.tkoihk(active, since 2s) 2026-02-13T10:37:16.727 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:16 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3060234602' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2026-02-13T10:37:16.727 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:16 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3711716976' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout { 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 5, 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "active_name": "trial121.tkoihk", 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_standby": 0 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout } 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for the mgr to restart... 2026-02-13T10:37:16.733 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for mgr epoch 5... 2026-02-13T10:37:17.572 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:17 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3711716976' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2026-02-13T10:37:17.573 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:17 trial121 ceph-mon[19159]: mgrmap e5: trial121.tkoihk(active, since 3s) 2026-02-13T10:37:17.573 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:17 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/300687823' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2026-02-13T10:37:19.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: Active manager daemon trial121.tkoihk restarted 2026-02-13T10:37:19.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: Activating manager daemon trial121.tkoihk 2026-02-13T10:37:19.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: osdmap e2: 0 total, 0 up, 0 in 2026-02-13T10:37:19.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: mgrmap e6: trial121.tkoihk(active, starting, since 0.00254851s) 2026-02-13T10:37:19.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr metadata", "who": "trial121.tkoihk", "id": "trial121.tkoihk"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: Manager daemon trial121.tkoihk is now available 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/mirror_snapshot_schedule"}]: dispatch 2026-02-13T10:37:19.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:19 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/trash_purge_schedule"}]: dispatch 2026-02-13T10:37:20.181 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout { 2026-02-13T10:37:20.181 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2026-02-13T10:37:20.181 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "initialized": true 2026-02-13T10:37:20.181 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout } 2026-02-13T10:37:20.181 INFO:teuthology.orchestra.run.trial121.stdout:mgr epoch 5 is available 2026-02-13T10:37:20.181 INFO:teuthology.orchestra.run.trial121.stdout:Setting orchestrator backend to cephadm... 2026-02-13T10:37:20.598 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout value unchanged 2026-02-13T10:37:20.598 INFO:teuthology.orchestra.run.trial121.stdout:Generating ssh key... 2026-02-13T10:37:20.707 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:20 trial121 ceph-mon[19159]: Found migration_current of "None". Setting to last migration. 2026-02-13T10:37:20.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:20 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:20.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:20 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:20.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:20 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:20.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:20 trial121 ceph-mon[19159]: mgrmap e7: trial121.tkoihk(active, since 1.00436s) 2026-02-13T10:37:20.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:20 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:20.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:20 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:21.019 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOh/UKLLJv5jdZ2VmFVgfmom5/8G8TezNup9Jv+dJ/Ix ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:21.019 INFO:teuthology.orchestra.run.trial121.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2026-02-13T10:37:21.019 INFO:teuthology.orchestra.run.trial121.stdout:Adding key to root@localhost authorized_keys... 2026-02-13T10:37:21.020 INFO:teuthology.orchestra.run.trial121.stdout:Adding host trial121... 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:19] ENGINE Bus STARTING 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:20] ENGINE Serving on https://10.20.193.121:7150 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:20] ENGINE Client ('10.20.193.121', 57882) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:20] ENGINE Serving on http://10.20.193.121:8765 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:20] ENGINE Bus STARTED 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:21.525 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:21 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:22.656 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:22 trial121 ceph-mon[19159]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:22.656 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:22 trial121 ceph-mon[19159]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:22.656 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:22 trial121 ceph-mon[19159]: Generating ssh key... 2026-02-13T10:37:22.656 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:22 trial121 ceph-mon[19159]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:22.656 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:22 trial121 ceph-mon[19159]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial121", "addr": "10.20.193.121", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:22.656 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:22 trial121 ceph-mon[19159]: mgrmap e8: trial121.tkoihk(active, since 2s) 2026-02-13T10:37:22.691 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Added host 'trial121' with addr '10.20.193.121' 2026-02-13T10:37:22.691 INFO:teuthology.orchestra.run.trial121.stdout:Deploying mon service with default placement... 2026-02-13T10:37:22.911 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2026-02-13T10:37:22.911 INFO:teuthology.orchestra.run.trial121.stdout:Deploying mgr service with default placement... 2026-02-13T10:37:23.125 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2026-02-13T10:37:23.125 INFO:teuthology.orchestra.run.trial121.stdout:Deploying crash service with default placement... 2026-02-13T10:37:23.347 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2026-02-13T10:37:23.347 INFO:teuthology.orchestra.run.trial121.stdout:Deploying ceph-exporter service with default placement... 2026-02-13T10:37:23.570 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2026-02-13T10:37:23.570 INFO:teuthology.orchestra.run.trial121.stdout:Deploying prometheus service with default placement... 2026-02-13T10:37:23.707 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: Deploying cephadm binary to trial121 2026-02-13T10:37:23.707 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:23.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:23.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:23.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:23.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:23.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:23.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:23 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:23.807 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2026-02-13T10:37:23.807 INFO:teuthology.orchestra.run.trial121.stdout:Deploying grafana service with default placement... 2026-02-13T10:37:24.039 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2026-02-13T10:37:24.039 INFO:teuthology.orchestra.run.trial121.stdout:Deploying node-exporter service with default placement... 2026-02-13T10:37:24.273 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2026-02-13T10:37:24.273 INFO:teuthology.orchestra.run.trial121.stdout:Deploying alertmanager service with default placement... 2026-02-13T10:37:24.487 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: Added host trial121 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: Saving service mon spec with placement count:5 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: Saving service mgr spec with placement count:2 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: Saving service crash spec with placement * 2026-02-13T10:37:24.601 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:24.602 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:24.602 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:24.602 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:24.602 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:24.602 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:24 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:24.908 INFO:teuthology.orchestra.run.trial121.stdout:Enabling the dashboard module... 2026-02-13T10:37:25.899 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:25.899 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: Saving service ceph-exporter spec with placement * 2026-02-13T10:37:25.899 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:25.899 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: Saving service prometheus spec with placement count:1 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: Saving service grafana spec with placement count:1 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: Saving service node-exporter spec with placement * 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: Saving service alertmanager spec with placement count:1 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='mgr.14118 10.20.193.121:0/3682352756' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3339577465' entity='client.admin' 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/4062950041' entity='client.admin' 2026-02-13T10:37:25.900 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:25 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2137907599' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout { 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "epoch": 9, 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "active_name": "trial121.tkoihk", 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "num_standby": 0 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout } 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for the mgr to restart... 2026-02-13T10:37:26.146 INFO:teuthology.orchestra.run.trial121.stdout:Waiting for mgr epoch 9... 2026-02-13T10:37:26.997 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:26 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2137907599' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2026-02-13T10:37:26.997 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:26 trial121 ceph-mon[19159]: mgrmap e9: trial121.tkoihk(active, since 6s) 2026-02-13T10:37:26.997 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:26 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/227413706' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: Active manager daemon trial121.tkoihk restarted 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: Activating manager daemon trial121.tkoihk 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: osdmap e3: 0 total, 0 up, 0 in 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: mgrmap e10: trial121.tkoihk(active, starting, since 0.00270073s) 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr metadata", "who": "trial121.tkoihk", "id": "trial121.tkoihk"}]: dispatch 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata"}]: dispatch 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata"}]: dispatch 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata"}]: dispatch 2026-02-13T10:37:28.933 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: Manager daemon trial121.tkoihk is now available 2026-02-13T10:37:28.934 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:28.934 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/mirror_snapshot_schedule"}]: dispatch 2026-02-13T10:37:28.934 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:28 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/trash_purge_schedule"}]: dispatch 2026-02-13T10:37:29.636 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout { 2026-02-13T10:37:29.636 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2026-02-13T10:37:29.636 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout "initialized": true 2026-02-13T10:37:29.636 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout } 2026-02-13T10:37:29.637 INFO:teuthology.orchestra.run.trial121.stdout:mgr epoch 9 is available 2026-02-13T10:37:29.637 INFO:teuthology.orchestra.run.trial121.stdout:Generating a dashboard self-signed certificate... 2026-02-13T10:37:29.925 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2026-02-13T10:37:29.925 INFO:teuthology.orchestra.run.trial121.stdout:Creating initial admin user... 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:29] ENGINE Bus STARTING 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:29] ENGINE Serving on https://10.20.193.121:7150 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:29] ENGINE Client ('10.20.193.121', 35252) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:29] ENGINE Serving on http://10.20.193.121:8765 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:29] ENGINE Bus STARTED 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: mgrmap e11: trial121.tkoihk(active, since 1.00441s) 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:30.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:30 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:30.280 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$FtK.B9p8vMbrD9XM0CfMNexf7.ZbJome3uTRYXJRjqhTm84Cv8lNC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1770979050, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2026-02-13T10:37:30.280 INFO:teuthology.orchestra.run.trial121.stdout:Fetching dashboard port number... 2026-02-13T10:37:30.486 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stdout 8443 2026-02-13T10:37:30.495 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-13T10:37:30.495 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout disabled 2026-02-13T10:37:30.501 INFO:teuthology.orchestra.run.trial121.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-13T10:37:30.501 INFO:teuthology.orchestra.run.trial121.stdout:systemctl: stdout inactive 2026-02-13T10:37:30.501 INFO:teuthology.orchestra.run.trial121.stdout:firewalld.service is not enabled 2026-02-13T10:37:30.501 INFO:teuthology.orchestra.run.trial121.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2026-02-13T10:37:30.502 INFO:teuthology.orchestra.run.trial121.stdout:Ceph Dashboard is now available at: 2026-02-13T10:37:30.502 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.502 INFO:teuthology.orchestra.run.trial121.stdout: URL: https://trial121:8443/ 2026-02-13T10:37:30.502 INFO:teuthology.orchestra.run.trial121.stdout: User: admin 2026-02-13T10:37:30.502 INFO:teuthology.orchestra.run.trial121.stdout: Password: fkv9oip9nx 2026-02-13T10:37:30.502 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.502 INFO:teuthology.orchestra.run.trial121.stdout:Saving cluster configuration to /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config directory 2026-02-13T10:37:30.761 INFO:teuthology.orchestra.run.trial121.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2026-02-13T10:37:30.761 INFO:teuthology.orchestra.run.trial121.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2026-02-13T10:37:30.761 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout:Or, if you are only running a single cluster on this host: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout:Please consider enabling telemetry to help improve Ceph: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: ceph telemetry on 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout:For more information see: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:30.762 INFO:teuthology.orchestra.run.trial121.stdout:Bootstrap complete. 2026-02-13T10:37:30.787 INFO:tasks.cephadm:Fetching config... 2026-02-13T10:37:30.787 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:30.787 DEBUG:teuthology.orchestra.run.trial121:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2026-02-13T10:37:30.803 INFO:tasks.cephadm:Fetching client.admin keyring... 2026-02-13T10:37:30.803 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:30.803 DEBUG:teuthology.orchestra.run.trial121:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2026-02-13T10:37:30.860 INFO:tasks.cephadm:Fetching mon keyring... 2026-02-13T10:37:30.861 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:30.861 DEBUG:teuthology.orchestra.run.trial121:> sudo dd if=/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/keyring of=/dev/stdout 2026-02-13T10:37:30.925 INFO:tasks.cephadm:Fetching pub ssh key... 2026-02-13T10:37:30.925 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:30.925 DEBUG:teuthology.orchestra.run.trial121:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2026-02-13T10:37:30.983 INFO:tasks.cephadm:Installing pub ssh key for root users... 2026-02-13T10:37:30.983 DEBUG:teuthology.orchestra.run.trial121:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOh/UKLLJv5jdZ2VmFVgfmom5/8G8TezNup9Jv+dJ/Ix ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-13T10:37:31.054 INFO:teuthology.orchestra.run.trial121.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOh/UKLLJv5jdZ2VmFVgfmom5/8G8TezNup9Jv+dJ/Ix ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:31.065 DEBUG:teuthology.orchestra.run.trial163:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOh/UKLLJv5jdZ2VmFVgfmom5/8G8TezNup9Jv+dJ/Ix ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-13T10:37:31.099 INFO:teuthology.orchestra.run.trial163.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOh/UKLLJv5jdZ2VmFVgfmom5/8G8TezNup9Jv+dJ/Ix ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:31.109 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2026-02-13T10:37:31.353 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:31 trial121 ceph-mon[19159]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:31.354 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:31 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:31.354 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:31 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/570370538' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2026-02-13T10:37:31.354 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:31 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1132670216' entity='client.admin' 2026-02-13T10:37:31.418 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:31.658 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2026-02-13T10:37:31.659 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch client-keyring set client.admin '*' --mode 0755 2026-02-13T10:37:31.947 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:32.217 INFO:tasks.cephadm:Writing (initial) conf and keyring to trial163 2026-02-13T10:37:32.217 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:37:32.217 DEBUG:teuthology.orchestra.run.trial163:> dd of=/etc/ceph/ceph.conf 2026-02-13T10:37:32.236 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:37:32.236 DEBUG:teuthology.orchestra.run.trial163:> dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:32.293 INFO:tasks.cephadm:Adding host trial163 to orchestrator... 2026-02-13T10:37:32.294 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch host add trial163 2026-02-13T10:37:32.590 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:32.613 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: mgrmap e12: trial121.tkoihk(active, since 2s) 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/721496101' entity='client.admin' 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd/host:trial121", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial121", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial121", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: Deploying daemon ceph-exporter.trial121 on trial121 2026-02-13T10:37:32.614 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:32 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.trial121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.trial121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: Deploying daemon crash.trial121 on trial121 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='client.14187 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial163", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:33.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:33 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:34.382 INFO:teuthology.orchestra.run.trial121.stdout:Added host 'trial163' with addr '10.20.193.163' 2026-02-13T10:37:34.419 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch host ls --format=json 2026-02-13T10:37:34.703 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:34.903 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:37:34.903 INFO:teuthology.orchestra.run.trial121.stdout:[{"addr": "10.20.193.121", "hostname": "trial121", "labels": [], "status": ""}, {"addr": "10.20.193.163", "hostname": "trial163", "labels": [], "status": ""}] 2026-02-13T10:37:34.903 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: Deploying daemon node-exporter.trial121 on trial121 2026-02-13T10:37:34.903 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: Deploying cephadm binary to trial163 2026-02-13T10:37:34.904 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:34.904 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:34.904 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:34.904 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:34.904 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:34.904 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:34 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:34.940 INFO:tasks.cephadm:Setting crush tunables to default 2026-02-13T10:37:34.940 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd crush tunables default 2026-02-13T10:37:35.274 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:35.696 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:35 trial121 ceph-mon[19159]: Added host trial163 2026-02-13T10:37:35.696 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:35 trial121 ceph-mon[19159]: Deploying daemon alertmanager.trial121 on trial121 2026-02-13T10:37:35.696 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:35 trial121 ceph-mon[19159]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T10:37:35.696 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:35 trial121 ceph-mon[19159]: mgrmap e13: trial121.tkoihk(active, since 6s) 2026-02-13T10:37:35.696 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:35 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1494479474' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2026-02-13T10:37:36.385 INFO:teuthology.orchestra.run.trial121.stderr:adjusted tunables profile to default 2026-02-13T10:37:36.426 INFO:tasks.cephadm:Adding mon.trial121 on trial121 2026-02-13T10:37:36.426 INFO:tasks.cephadm:Adding mon.trial163 on trial163 2026-02-13T10:37:36.427 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch apply mon '2;trial121:10.20.193.121=trial121;trial163:10.20.193.163=trial163' 2026-02-13T10:37:36.707 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:36.735 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:36.940 INFO:teuthology.orchestra.run.trial163.stdout:Scheduled mon update... 2026-02-13T10:37:36.977 DEBUG:teuthology.orchestra.run.trial163:mon.trial163> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial163.service 2026-02-13T10:37:36.979 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:36.979 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:37.292 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:37.320 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: Regenerating cephadm self-signed grafana TLS certificates 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: Deploying daemon grafana.trial121 on trial121 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1494479474' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: osdmap e4: 0 total, 0 up, 0 in 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;trial121:10.20.193.121=trial121;trial163:10.20.193.163=trial163", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:37:37.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:36 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:37.536 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:37.536 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:37.537 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:38.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:37 trial121 ceph-mon[19159]: Saving service mon spec with placement trial121:10.20.193.121=trial121;trial163:10.20.193.163=trial163;count:2 2026-02-13T10:37:38.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:37 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/3544055332' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:38.572 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:38.572 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:38.847 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:38.875 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:39.098 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:39.099 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:39.099 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1523519609' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:39.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:39 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:40.155 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:40.155 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:40.428 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:40.458 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:40.682 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:40.682 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:40.683 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:40.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:40 trial121 ceph-mon[19159]: Deploying daemon prometheus.trial121 on trial121 2026-02-13T10:37:41.738 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:41.738 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:41.815 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:41 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/4211803282' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:42.012 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:42.041 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:42.267 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:42.267 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:42.267 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:43.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:42 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:43.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:42 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:43.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:42 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:43.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:42 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2026-02-13T10:37:43.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:42 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/2261029347' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:43.302 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:43.302 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:43.578 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:43.608 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:43.829 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:43.829 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:43.829 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:44.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:43 trial121 ceph-mon[19159]: from='mgr.14162 10.20.193.121:0/2112862316' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2026-02-13T10:37:44.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:43 trial121 ceph-mon[19159]: mgrmap e14: trial121.tkoihk(active, since 14s) 2026-02-13T10:37:44.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:43 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1978969784' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:44.868 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:44.868 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:45.143 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:45.173 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:45.397 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:45.397 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:45.398 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:45.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:45 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/3519484429' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:46.433 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:46.433 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:46.541 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: Active manager daemon trial121.tkoihk restarted 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: Activating manager daemon trial121.tkoihk 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: osdmap e5: 0 total, 0 up, 0 in 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: mgrmap e15: trial121.tkoihk(active, starting, since 0.00271984s) 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr metadata", "who": "trial121.tkoihk", "id": "trial121.tkoihk"}]: dispatch 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata"}]: dispatch 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata"}]: dispatch 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata"}]: dispatch 2026-02-13T10:37:46.542 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: Manager daemon trial121.tkoihk is now available 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/mirror_snapshot_schedule"}]: dispatch 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial121.tkoihk/trash_purge_schedule"}]: dispatch 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:46.543 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:46.772 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:46.808 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T10:37:47.030 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:47.030 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:47.030 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:47.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:46] ENGINE Bus STARTING 2026-02-13T10:37:47.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:46] ENGINE Serving on https://10.20.193.121:7150 2026-02-13T10:37:47.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:46] ENGINE Client ('10.20.193.121', 36270) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:46] ENGINE Serving on http://10.20.193.121:8765 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: [13/Feb/2026:10:37:46] ENGINE Bus STARTED 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: mgrmap e16: trial121.tkoihk(active, since 1.00688s) 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd/host:trial121", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1627850176' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:47.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd/host:trial163", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:37:47.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:47.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:37:48.071 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:48.071 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:48.393 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.conf 2026-02-13T10:37:48.728 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:48.728 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:48.728 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial121:/etc/ceph/ceph.conf 2026-02-13T10:37:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial163:/etc/ceph/ceph.conf 2026-02-13T10:37:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial163:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.conf 2026-02-13T10:37:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial121:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.conf 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial163:/etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial121:/etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial163:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Updating trial121:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2026-02-13T10:37:48.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:48.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: Deploying daemon ceph-exporter.trial163 on trial163 2026-02-13T10:37:48.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:48 trial121 ceph-mon[19159]: mgrmap e17: trial121.tkoihk(active, since 2s) 2026-02-13T10:37:49.767 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:49.768 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:50.054 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.conf 2026-02-13T10:37:50.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.trial163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/4200952048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.trial163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: Deploying daemon crash.trial163 on trial163 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:49 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:50.289 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:50.289 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":1,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:08.375508Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2026-02-13T10:37:50.289 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 1 2026-02-13T10:37:51.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: Deploying daemon node-exporter.trial163 on trial163 2026-02-13T10:37:51.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1143916877' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:51.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:51.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:51.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:51.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:51.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.trial163.xfyaeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2026-02-13T10:37:51.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.trial163.xfyaeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-13T10:37:51.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr services"}]: dispatch 2026-02-13T10:37:51.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:51.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:50 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:51.403 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2026-02-13T10:37:51.404 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T10:37:51.727 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:37:52.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:51 trial121 ceph-mon[19159]: Deploying daemon mgr.trial163.xfyaeo on trial163 2026-02-13T10:37:52.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:51 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:52.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:51 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:52.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:51 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:52.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:51 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:52.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:51 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2026-02-13T10:37:52.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:51 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:56.961 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:37:56.962 INFO:teuthology.orchestra.run.trial163.stdout:{"epoch":2,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","modified":"2026-02-13T10:37:51.952847Z","created":"2026-02-13T10:37:08.375508Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"trial121","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:3300","nonce":0},{"type":"v1","addr":"10.20.193.121:6789","nonce":0}]},"addr":"10.20.193.121:6789/0","public_addr":"10.20.193.121:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"trial163","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:3300","nonce":0},{"type":"v1","addr":"10.20.193.163:6789","nonce":0}]},"addr":"10.20.193.163:6789/0","public_addr":"10.20.193.163:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2026-02-13T10:37:56.962 INFO:teuthology.orchestra.run.trial163.stderr:dumped monmap epoch 2 2026-02-13T10:37:57.023 INFO:tasks.cephadm:Generating final ceph.conf file... 2026-02-13T10:37:57.024 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config generate-minimal-conf 2026-02-13T10:37:57.307 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:57.330 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: mon.trial121 calling monitor election 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1291792022' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.? 10.20.193.163:0/3566833856' entity='mgr.trial163.xfyaeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/trial163.xfyaeo/crt"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: mon.trial163 calling monitor election 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:57.331 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: mon.trial121 is new leader, mons trial121,trial163 in quorum (ranks 0,1) 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: monmap epoch 2 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: last_changed 2026-02-13T10:37:51.952847+0000 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: created 2026-02-13T10:37:08.375508+0000 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: min_mon_release 19 (squid) 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: election_strategy: 1 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: 0: [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] mon.trial121 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: 1: [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] mon.trial163 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: fsmap 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: osdmap e5: 0 total, 0 up, 0 in 2026-02-13T10:37:57.332 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: mgrmap e17: trial121.tkoihk(active, since 11s) 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: overall HEALTH_OK 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: Standby manager daemon trial163.xfyaeo started 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.? 10.20.193.163:0/3566833856' entity='mgr.trial163.xfyaeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.? 10.20.193.163:0/3566833856' entity='mgr.trial163.xfyaeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/trial163.xfyaeo/key"}]: dispatch 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.? 10.20.193.163:0/3566833856' entity='mgr.trial163.xfyaeo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:57.333 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:37:57.499 INFO:teuthology.orchestra.run.trial121.stdout:# minimal ceph.conf for e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:57.499 INFO:teuthology.orchestra.run.trial121.stdout:[global] 2026-02-13T10:37:57.499 INFO:teuthology.orchestra.run.trial121.stdout: fsid = e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:37:57.499 INFO:teuthology.orchestra.run.trial121.stdout: mon_host = [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] 2026-02-13T10:37:57.551 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2026-02-13T10:37:57.551 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:57.552 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/etc/ceph/ceph.conf 2026-02-13T10:37:57.578 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:57.578 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:57.643 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:37:57.643 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/etc/ceph/ceph.conf 2026-02-13T10:37:57.670 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:37:57.670 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T10:37:57.735 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2026-02-13T10:37:57.735 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set client client_mount_timeout 600 2026-02-13T10:37:57.738 INFO:tasks.cephadm:setting config [client] debug_client = 20 2026-02-13T10:37:57.738 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set client debug_client 20 2026-02-13T10:37:57.781 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2026-02-13T10:37:57.782 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set client debug_ms 1 2026-02-13T10:37:57.825 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2026-02-13T10:37:57.825 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set client rados_mon_op_timeout 900 2026-02-13T10:37:57.869 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2026-02-13T10:37:57.869 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set client rados_osd_op_timeout 900 2026-02-13T10:37:57.912 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2026-02-13T10:37:57.912 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds debug_mds 20 2026-02-13T10:37:57.956 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2026-02-13T10:37:57.956 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds debug_mds_balancer 20 2026-02-13T10:37:57.999 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2026-02-13T10:37:57.999 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds debug_ms 2 2026-02-13T10:37:58.034 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.035 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2026-02-13T10:37:58.035 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds mds_debug_frag True 2026-02-13T10:37:58.079 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2026-02-13T10:37:58.079 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds mds_debug_scatterstat True 2026-02-13T10:37:58.100 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.102 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2026-02-13T10:37:58.102 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds mds_op_complaint_time 180 2026-02-13T10:37:58.144 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2026-02-13T10:37:58.144 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds mds_verify_scatter True 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: mgrmap e18: trial121.tkoihk(active, since 11s), standbys: trial163.xfyaeo 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr metadata", "who": "trial163.xfyaeo", "id": "trial163.xfyaeo"}]: dispatch 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: Updating trial121:/etc/ceph/ceph.conf 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: Updating trial163:/etc/ceph/ceph.conf 2026-02-13T10:37:58.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/445310556' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: Updating trial163:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.conf 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: Updating trial121:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/config/ceph.conf 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: Reconfiguring mon.trial121 (unknown last config time)... 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: Reconfiguring daemon mon.trial121 on trial121 2026-02-13T10:37:58.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:37:58.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:58.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.trial121.tkoihk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2026-02-13T10:37:58.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr services"}]: dispatch 2026-02-13T10:37:58.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:58.172 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2026-02-13T10:37:58.172 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds osd_op_complaint_time 180 2026-02-13T10:37:58.180 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.181 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2026-02-13T10:37:58.182 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds rados_mon_op_timeout 900 2026-02-13T10:37:58.224 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2026-02-13T10:37:58.224 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mds rados_osd_op_timeout 900 2026-02-13T10:37:58.245 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.246 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2026-02-13T10:37:58.247 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mgr client_mount_timeout 30 2026-02-13T10:37:58.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:58.290 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2026-02-13T10:37:58.291 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mgr debug_client 20 2026-02-13T10:37:58.301 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.302 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2026-02-13T10:37:58.302 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mgr debug_mgr 20 2026-02-13T10:37:58.334 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2026-02-13T10:37:58.334 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mgr debug_ms 1 2026-02-13T10:37:58.372 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.374 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2026-02-13T10:37:58.374 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mgr mon_warn_on_pool_no_app False 2026-02-13T10:37:58.412 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.414 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2026-02-13T10:37:58.414 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set mon mon_op_complaint_time 120 2026-02-13T10:37:58.450 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.447+0000 7fdebf262640 1 -- 10.20.193.121:0/2260507496 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 msgr2=0x7fdeb8101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.450 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.447+0000 7fdebf262640 1 --2- 10.20.193.121:0/2260507496 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb8101960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdea80098e0 tx=0x7fdea802f1d0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.450 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.447+0000 7fdebf262640 1 -- 10.20.193.121:0/2260507496 shutdown_connections 2026-02-13T10:37:58.450 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.447+0000 7fdebf262640 1 --2- 10.20.193.121:0/2260507496 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fdeb8102760 0x7fdeb8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.450 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.447+0000 7fdebf262640 1 --2- 10.20.193.121:0/2260507496 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb8101960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.450 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.447+0000 7fdebf262640 1 -- 10.20.193.121:0/2260507496 >> 10.20.193.121:0/2260507496 conn(0x7fdeb80fcd10 msgr2=0x7fdeb80ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.450 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.447+0000 7fdebf262640 1 -- 10.20.193.121:0/2260507496 shutdown_connections 2026-02-13T10:37:58.451 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.448+0000 7fdebf262640 1 -- 10.20.193.121:0/2260507496 wait complete. 2026-02-13T10:37:58.452 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2026-02-13T10:37:58.452 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config set osd osd_op_complaint_time 180 2026-02-13T10:37:58.453 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.448+0000 7f76d076f640 1 -- 10.20.193.121:0/1934310316 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f76a40774c0 msgr2=0x7f76a4079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.453 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.448+0000 7f76d076f640 1 --2- 10.20.193.121:0/1934310316 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f76a40774c0 0x7f76a4079980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76c8111090 tx=0x7f76b8004050 comp rx=0 tx=0).stop 2026-02-13T10:37:58.453 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.448+0000 7f76d076f640 1 -- 10.20.193.121:0/1934310316 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f76c8100930 msgr2=0x7f76c81ab820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.453 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.448+0000 7f76d076f640 1 --2- 10.20.193.121:0/1934310316 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f76c8100930 0x7f76c81ab820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f76bc02f6e0 tx=0x7f76bc02ff70 comp rx=0 tx=0).stop 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.449+0000 7fdebf262640 1 Processor -- start 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.449+0000 7fdebf262640 1 -- start start 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.449+0000 7fdebf262640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebf262640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fdeb8102760 0x7fdeb819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebf262640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb810b110 con 0x7fdeb8101560 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebf262640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb810af90 con 0x7fdeb8102760 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebcfd7640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebcfd7640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56992/0 (socket says 10.20.193.121:56992) 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebcfd7640 1 -- 10.20.193.121:0/560264047 learned_addr learned my addr 10.20.193.121:0/560264047 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebcfd7640 1 -- 10.20.193.121:0/560264047 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fdeb8102760 msgr2=0x7fdeb819a5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebcfd7640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fdeb8102760 0x7fdeb819a5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebcfd7640 1 -- 10.20.193.121:0/560264047 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb819ac30 con 0x7fdeb8101560 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdebcfd7640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb819a070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdea802f6e0 tx=0x7fdea802ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.450+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fdea8046070 con 0x7fdeb8101560 2026-02-13T10:37:58.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(24 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fdea80044e0 con 0x7fdeb8101560 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7f76d076f640 1 -- 10.20.193.121:0/1934310316 shutdown_connections 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7f76d076f640 1 --2- 10.20.193.121:0/1934310316 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f76a40774c0 0x7f76a4079980 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7f76d076f640 1 --2- 10.20.193.121:0/1934310316 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f76c8103990 0x7f76c81134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7f76d076f640 1 --2- 10.20.193.121:0/1934310316 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f76c8100930 0x7f76c81ab820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7f76d076f640 1 -- 10.20.193.121:0/1934310316 >> 10.20.193.121:0/1934310316 conn(0x7f76c80fc460 msgr2=0x7f76c8106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7f76d076f640 1 -- 10.20.193.121:0/1934310316 shutdown_connections 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fdea8041630 con 0x7fdeb8101560 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb819f610 con 0x7fdeb8101560 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.451+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb819fb70 con 0x7fdeb8101560 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.452+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb810b8f0 con 0x7fdeb8101560 2026-02-13T10:37:58.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.453+0000 7f76d076f640 1 -- 10.20.193.121:0/1934310316 wait complete. 2026-02-13T10:37:58.522 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.453+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fdea8038470 con 0x7fdeb8101560 2026-02-13T10:37:58.522 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.454+0000 7fdeadffb640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fde94077350 0x7fde94079810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.522 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.454+0000 7fdeaffff640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fde94077350 0x7fde94079810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.522 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.454+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fdea803e070 con 0x7fdeb8101560 2026-02-13T10:37:58.522 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.455+0000 7fdeaffff640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fde94077350 0x7fde94079810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdea00059c0 tx=0x7fdea0005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.522 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.458+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdea808adf0 con 0x7fdeb8101560 2026-02-13T10:37:58.522 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.523 DEBUG:tasks.cephadm:set 22 configs 2026-02-13T10:37:58.524 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.524 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.524 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.540 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.551 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.557 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.555+0000 7f340cb06640 1 -- 10.20.193.121:0/2793272989 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f34081a1fa0 con 0x7f3408107c50 2026-02-13T10:37:58.559 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.561 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.559+0000 7f33eb7fe640 1 -- 10.20.193.121:0/2793272989 <== mon.0 v2:10.20.193.121:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f33f4004ae0 con 0x7f3408107c50 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 -- 10.20.193.121:0/2222522091 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844109f90 msgr2=0x7f284410a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 --2- 10.20.193.121:0/2222522091 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844109f90 0x7f284410a410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28300099b0 tx=0x7f283002f240 comp rx=0 tx=0).stop 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 -- 10.20.193.121:0/2222522091 shutdown_connections 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 --2- 10.20.193.121:0/2222522091 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844109f90 0x7f284410a410 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 --2- 10.20.193.121:0/2222522091 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2844108e30 0x7f2844109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 -- 10.20.193.121:0/2222522091 >> 10.20.193.121:0/2222522091 conn(0x7f2844077de0 msgr2=0x7f284407a220 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 -- 10.20.193.121:0/2222522091 shutdown_connections 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.562+0000 7f284a83c640 1 -- 10.20.193.121:0/2222522091 wait complete. 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f284a83c640 1 Processor -- start 2026-02-13T10:37:58.564 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f284a83c640 1 -- start start 2026-02-13T10:37:58.566 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f284a83c640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844108e30 0x7f28440831b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.566 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f284a83c640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2844109f90 0x7f28440836f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.566 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f284a83c640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2844111940 con 0x7f2844108e30 2026-02-13T10:37:58.566 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f284a83c640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f28441117c0 con 0x7f2844109f90 2026-02-13T10:37:58.566 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f2843fff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844108e30 0x7f28440831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.566 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f2843fff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844108e30 0x7f28440831b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57020/0 (socket says 10.20.193.121:57020) 2026-02-13T10:37:58.567 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f2843fff640 1 -- 10.20.193.121:0/2361331655 learned_addr learned my addr 10.20.193.121:0/2361331655 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.567 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f28437fe640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2844109f90 0x7f28440836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.567 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f2843fff640 1 -- 10.20.193.121:0/2361331655 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2844109f90 msgr2=0x7f28440836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.567 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f2843fff640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2844109f90 0x7f28440836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.567 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f2843fff640 1 -- 10.20.193.121:0/2361331655 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2844083d70 con 0x7f2844108e30 2026-02-13T10:37:58.567 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f340cb06640 1 -- 10.20.193.121:0/2793272989 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f33e00774c0 msgr2=0x7f33e0079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.567 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f340cb06640 1 --2- 10.20.193.121:0/2793272989 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f33e00774c0 0x7f33e0079980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0040c0 tx=0x7f33fc004800 comp rx=0 tx=0).stop 2026-02-13T10:37:58.568 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f340cb06640 1 -- 10.20.193.121:0/2793272989 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3408107c50 msgr2=0x7f340819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.568 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f340cb06640 1 --2- 10.20.193.121:0/2793272989 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3408107c50 0x7f340819c6a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f33f400d900 tx=0x7f33f400ddd0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.568 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.563+0000 7f28437fe640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2844109f90 0x7f28440836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:37:58.568 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f2843fff640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844108e30 0x7f28440831b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f283400c910 tx=0x7f283400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.569 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 7 ==== config(25 keys) ==== 1034+0+0 (secure 0 0 0) 0x7fdea80c2080 con 0x7fdeb8101560 2026-02-13T10:37:58.569 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f2834007c30 con 0x7f2844108e30 2026-02-13T10:37:58.570 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(25 keys) ==== 1034+0+0 (secure 0 0 0) 0x7f2834007dd0 con 0x7f2844108e30 2026-02-13T10:37:58.570 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28441ae710 con 0x7f2844108e30 2026-02-13T10:37:58.570 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f283400f450 con 0x7f2844108e30 2026-02-13T10:37:58.570 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441aecd0 con 0x7f2844108e30 2026-02-13T10:37:58.570 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3283348117 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01026d0 msgr2=0x7f29e0102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.571 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.564+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3283348117 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01026d0 0x7f29e0102ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29d40099b0 tx=0x7f29d402f220 comp rx=0 tx=0).stop 2026-02-13T10:37:58.571 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3283348117 shutdown_connections 2026-02-13T10:37:58.571 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3283348117 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f29e01038d0 0x7f29e0103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.571 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3283348117 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01026d0 0x7f29e0102ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.571 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3283348117 >> 10.20.193.121:0/3283348117 conn(0x7f29e00fde60 msgr2=0x7f29e01002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.571 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3283348117 shutdown_connections 2026-02-13T10:37:58.571 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3283348117 wait complete. 2026-02-13T10:37:58.572 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f340cb06640 1 -- 10.20.193.121:0/2793272989 shutdown_connections 2026-02-13T10:37:58.572 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f340cb06640 1 --2- 10.20.193.121:0/2793272989 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f33e00774c0 0x7f33e0079980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.572 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f340cb06640 1 --2- 10.20.193.121:0/2793272989 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3408107c50 0x7f340819c6a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.572 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f340cb06640 1 --2- 10.20.193.121:0/2793272989 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f340806b900 0x7f340819c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.572 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f340cb06640 1 -- 10.20.193.121:0/2793272989 >> 10.20.193.121:0/2793272989 conn(0x7f34080fcd20 msgr2=0x7f34080ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.572 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f340cb06640 1 -- 10.20.193.121:0/2793272989 shutdown_connections 2026-02-13T10:37:58.573 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 Processor -- start 2026-02-13T10:37:58.573 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 -- start start 2026-02-13T10:37:58.573 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284407cbc0 con 0x7f2844108e30 2026-02-13T10:37:58.574 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f29e01026d0 0x7f29e019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.574 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01038d0 0x7f29e019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.574 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f29e010b280 con 0x7f29e01038d0 2026-02-13T10:37:58.574 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.565+0000 7f29e4f9e640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f29e010b100 con 0x7f29e01026d0 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29ddd74640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01038d0 0x7f29e019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29ddd74640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01038d0 0x7f29e019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57024/0 (socket says 10.20.193.121:57024) 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29ddd74640 1 -- 10.20.193.121:0/3869377632 learned_addr learned my addr 10.20.193.121:0/3869377632 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29de575640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f29e01026d0 0x7f29e019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29ddd74640 1 -- 10.20.193.121:0/3869377632 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f29e01026d0 msgr2=0x7f29e019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29ddd74640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f29e01026d0 0x7f29e019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29ddd74640 1 -- 10.20.193.121:0/3869377632 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e019ace0 con 0x7f29e01038d0 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29ddd74640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01038d0 0x7f29e019a660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29c800d900 tx=0x7f29c800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f29c8014070 con 0x7f29e01038d0 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e019f720 con 0x7f29e01038d0 2026-02-13T10:37:58.575 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(25 keys) ==== 1034+0+0 (secure 0 0 0) 0x7f29c80044a0 con 0x7f29e01038d0 2026-02-13T10:37:58.576 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f29c8005020 con 0x7f29e01038d0 2026-02-13T10:37:58.576 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.566+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e019fd30 con 0x7f29e01038d0 2026-02-13T10:37:58.576 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.567+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f28340047c0 con 0x7f2844108e30 2026-02-13T10:37:58.576 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.567+0000 7f340cb06640 1 -- 10.20.193.121:0/2793272989 wait complete. 2026-02-13T10:37:58.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.567+0000 7f28417fa640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2818077510 0x7f28180799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.567+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f29c80047c0 con 0x7f29e01038d0 2026-02-13T10:37:58.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.567+0000 7f28437fe640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2818077510 0x7f28180799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.567+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f2834098700 con 0x7f2844108e30 2026-02-13T10:37:58.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.568+0000 7f28437fe640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2818077510 0x7f28180799d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28300097c0 tx=0x7f2830005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.568+0000 7f29c77fe640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f29b8077560 0x7f29b8079a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.568+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f29c8098580 con 0x7f29e01038d0 2026-02-13T10:37:58.579 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.568+0000 7f29de575640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f29b8077560 0x7f29b8079a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.579 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.568+0000 7f29de575640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f29b8077560 0x7f29b8079a20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f29d40097c0 tx=0x7f29d403a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.579 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.569+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e0102ad0 con 0x7f29e01038d0 2026-02-13T10:37:58.579 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.572+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2834014030 con 0x7f2844108e30 2026-02-13T10:37:58.579 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.577+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f29c8066710 con 0x7f29e01038d0 2026-02-13T10:37:58.594 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.592+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fdeb819fe20 con 0x7fdeb8101560 2026-02-13T10:37:58.600 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.598+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7fdea8082de0 con 0x7fdeb8101560 2026-02-13T10:37:58.600 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.598+0000 7fdeadffb640 1 -- 10.20.193.121:0/560264047 <== mon.0 v2:10.20.193.121:3300/0 9 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fdea8082a80 con 0x7fdeb8101560 2026-02-13T10:37:58.600 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.598+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 7 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f2834066890 con 0x7f2844108e30 2026-02-13T10:37:58.600 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.598+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 7 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f29c806b5c0 con 0x7f29e01038d0 2026-02-13T10:37:58.603 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fde94077350 msgr2=0x7fde94079810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.603 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fde94077350 0x7fde94079810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdea00059c0 tx=0x7fdea0005950 comp rx=0 tx=0).stop 2026-02-13T10:37:58.603 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 msgr2=0x7fdeb819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.603 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb819a070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdea802f6e0 tx=0x7fdea802ff90 comp rx=0 tx=0).stop 2026-02-13T10:37:58.603 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 shutdown_connections 2026-02-13T10:37:58.603 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fde94077350 0x7fde94079810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.604 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fdeb8102760 0x7fdeb819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.604 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 --2- 10.20.193.121:0/560264047 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fdeb8101560 0x7fdeb819a070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.604 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.601+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 >> 10.20.193.121:0/560264047 conn(0x7fdeb80fcd10 msgr2=0x7fdeb8105990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.604 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.602+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 shutdown_connections 2026-02-13T10:37:58.604 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.602+0000 7fdebf262640 1 -- 10.20.193.121:0/560264047 wait complete. 2026-02-13T10:37:58.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 -- 10.20.193.121:0/1409355081 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4104940 msgr2=0x7f75f4104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 --2- 10.20.193.121:0/1409355081 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4104940 0x7f75f4104d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f75e40099b0 tx=0x7f75e402f240 comp rx=0 tx=0).stop 2026-02-13T10:37:58.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 -- 10.20.193.121:0/1409355081 shutdown_connections 2026-02-13T10:37:58.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 --2- 10.20.193.121:0/1409355081 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4105b40 0x7f75f4105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 --2- 10.20.193.121:0/1409355081 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4104940 0x7f75f4104d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 -- 10.20.193.121:0/1409355081 >> 10.20.193.121:0/1409355081 conn(0x7f75f41000f0 msgr2=0x7f75f4102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 -- 10.20.193.121:0/1409355081 shutdown_connections 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.661+0000 7f75fab17640 1 -- 10.20.193.121:0/1409355081 wait complete. 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75fab17640 1 Processor -- start 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75fab17640 1 -- start start 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75fab17640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 0x7f75f419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75fab17640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4105b40 0x7f75f419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75fab17640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75f410d6b0 con 0x7f75f4105b40 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75fab17640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75f410d530 con 0x7f75f4104940 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75ebfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4105b40 0x7f75f419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75ebfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4105b40 0x7f75f419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57044/0 (socket says 10.20.193.121:57044) 2026-02-13T10:37:58.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75ebfff640 1 -- 10.20.193.121:0/3671126764 learned_addr learned my addr 10.20.193.121:0/3671126764 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75f888c640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 0x7f75f419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75f888c640 1 -- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 msgr2=0x7f75f419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75f888c640 1 -- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 msgr2=0x7f75f419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75f888c640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 0x7f75f419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75f888c640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 0x7f75f419c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75ebfff640 1 -- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 msgr2=0x7f75f419c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75ebfff640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 0x7f75f419c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.662+0000 7f75ebfff640 1 -- 10.20.193.121:0/3671126764 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f419d110 con 0x7f75f4105b40 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.663+0000 7f75ebfff640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4105b40 0x7f75f419ca90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b4f0 tx=0x7f75dc00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.663+0000 7f75e9ffb640 1 -- 10.20.193.121:0/3671126764 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f75dc004440 con 0x7f75f4105b40 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.663+0000 7f75e9ffb640 1 -- 10.20.193.121:0/3671126764 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f75dc015070 con 0x7f75f4105b40 2026-02-13T10:37:58.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.663+0000 7f75e9ffb640 1 -- 10.20.193.121:0/3671126764 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f75dc010bd0 con 0x7f75f4105b40 2026-02-13T10:37:58.668 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.663+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f41a1b50 con 0x7f75f4105b40 2026-02-13T10:37:58.668 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.663+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f41a2100 con 0x7f75f4105b40 2026-02-13T10:37:58.669 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.664+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f4104dc0 con 0x7f75f4105b40 2026-02-13T10:37:58.669 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.664+0000 7f75e9ffb640 1 -- 10.20.193.121:0/3671126764 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f75dc01a460 con 0x7f75f4105b40 2026-02-13T10:37:58.669 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.665+0000 7f75e9ffb640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f75d0077510 0x7f75d00799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.669 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.665+0000 7f75e9ffb640 1 -- 10.20.193.121:0/3671126764 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f75dc0984d0 con 0x7f75f4105b40 2026-02-13T10:37:58.669 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.665+0000 7f75f888c640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f75d0077510 0x7f75d00799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.669 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.665+0000 7f75f888c640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f75d0077510 0x7f75d00799d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75e4002c50 tx=0x7f75e4009680 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.669 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.667+0000 7f75e9ffb640 1 -- 10.20.193.121:0/3671126764 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f75dc0670d0 con 0x7f75f4105b40 2026-02-13T10:37:58.682 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.682 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.679+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f284407ce00 con 0x7f2844108e30 2026-02-13T10:37:58.685 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.684+0000 7f28417fa640 1 -- 10.20.193.121:0/2361331655 <== mon.0 v2:10.20.193.121:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7f283405e880 con 0x7f2844108e30 2026-02-13T10:37:58.687 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.685+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f29e0193a40 con 0x7f29e01038d0 2026-02-13T10:37:58.690 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.688+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2818077510 msgr2=0x7f28180799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.690 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.688+0000 7f284a83c640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2818077510 0x7f28180799d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f28300097c0 tx=0x7f2830005e50 comp rx=0 tx=0).stop 2026-02-13T10:37:58.691 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.689+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844108e30 msgr2=0x7f28440831b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.691 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.689+0000 7f284a83c640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844108e30 0x7f28440831b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f283400c910 tx=0x7f283400cde0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.691 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.688+0000 7f29c77fe640 1 -- 10.20.193.121:0/3869377632 <== mon.0 v2:10.20.193.121:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v17)=0 v17) ==== 117+0+0 (secure 0 0 0) 0x7f29c805e700 con 0x7f29e01038d0 2026-02-13T10:37:58.691 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.690+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 shutdown_connections 2026-02-13T10:37:58.691 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.690+0000 7f284a83c640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2818077510 0x7f28180799d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.692 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.690+0000 7f284a83c640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2844109f90 0x7f28440836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.692 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.690+0000 7f284a83c640 1 --2- 10.20.193.121:0/2361331655 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2844108e30 0x7f28440831b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.692 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.690+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 >> 10.20.193.121:0/2361331655 conn(0x7f2844077de0 msgr2=0x7f284410d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.694 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.692+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f29b8077560 msgr2=0x7f29b8079a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.694 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.692+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f29b8077560 0x7f29b8079a20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f29d40097c0 tx=0x7f29d403a040 comp rx=0 tx=0).stop 2026-02-13T10:37:58.694 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.692+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01038d0 msgr2=0x7f29e019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.694 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.692+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01038d0 0x7f29e019a660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29c800d900 tx=0x7f29c800ddd0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.695 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.693+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 shutdown_connections 2026-02-13T10:37:58.695 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.693+0000 7f284a83c640 1 -- 10.20.193.121:0/2361331655 wait complete. 2026-02-13T10:37:58.697 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.696+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 shutdown_connections 2026-02-13T10:37:58.697 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.696+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f29b8077560 0x7f29b8079a20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.697 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.696+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f29e01038d0 0x7f29e019a660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.698 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.696+0000 7f29e4f9e640 1 --2- 10.20.193.121:0/3869377632 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f29e01026d0 0x7f29e019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.698 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.696+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 >> 10.20.193.121:0/3869377632 conn(0x7f29e00fde60 msgr2=0x7f29e0107010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.699 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.698+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 shutdown_connections 2026-02-13T10:37:58.701 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.699+0000 7f29e4f9e640 1 -- 10.20.193.121:0/3869377632 wait complete. 2026-02-13T10:37:58.705 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.706 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.726 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 -- 10.20.193.121:0/2973832782 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c105b40 msgr2=0x7fbb9c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.726 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 --2- 10.20.193.121:0/2973832782 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c105b40 0x7fbb9c105fc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbb840099b0 tx=0x7fbb8402f220 comp rx=0 tx=0).stop 2026-02-13T10:37:58.726 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 -- 10.20.193.121:0/2973832782 shutdown_connections 2026-02-13T10:37:58.726 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 --2- 10.20.193.121:0/2973832782 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c105b40 0x7fbb9c105fc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.726 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 --2- 10.20.193.121:0/2973832782 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c104940 0x7fbb9c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.726 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 -- 10.20.193.121:0/2973832782 >> 10.20.193.121:0/2973832782 conn(0x7fbb9c1000f0 msgr2=0x7fbb9c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 -- 10.20.193.121:0/2973832782 shutdown_connections 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.722+0000 7fbba1740640 1 -- 10.20.193.121:0/2973832782 wait complete. 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbba1740640 1 Processor -- start 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbba1740640 1 -- start start 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbba1740640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c104940 0x7fbb9c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbba1740640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 0x7fbb9c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbba1740640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10d630 con 0x7fbb9c104940 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbba1740640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10d4b0 con 0x7fbb9c105b40 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c104940 0x7fbb9c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c104940 0x7fbb9c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57066/0 (socket says 10.20.193.121:57066) 2026-02-13T10:37:58.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9affd640 1 -- 10.20.193.121:0/1419093118 learned_addr learned my addr 10.20.193.121:0/1419093118 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9a7fc640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 0x7fbb9c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9a7fc640 1 -- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 msgr2=0x7fbb9c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9a7fc640 1 -- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 msgr2=0x7fbb9c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9a7fc640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 0x7fbb9c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9affd640 1 -- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 msgr2=0x7fbb9c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9a7fc640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 0x7fbb9c1984b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9affd640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 0x7fbb9c1984b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.723+0000 7fbb9affd640 1 -- 10.20.193.121:0/1419093118 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c1989f0 con 0x7fbb9c104940 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.724+0000 7fbb9affd640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c104940 0x7fbb9c197f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000d990 tx=0x7fbb9000de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.724+0000 7fbb7bfff640 1 -- 10.20.193.121:0/1419093118 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fbb90014070 con 0x7fbb9c104940 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.724+0000 7fbb7bfff640 1 -- 10.20.193.121:0/1419093118 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fbb9000b900 con 0x7fbb9c104940 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.724+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c06c890 con 0x7fbb9c104940 2026-02-13T10:37:58.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.724+0000 7fbb7bfff640 1 -- 10.20.193.121:0/1419093118 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fbb90002d70 con 0x7fbb9c104940 2026-02-13T10:37:58.729 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.724+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c06cea0 con 0x7fbb9c104940 2026-02-13T10:37:58.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.725+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c10de10 con 0x7fbb9c104940 2026-02-13T10:37:58.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.725+0000 7fbb7bfff640 1 -- 10.20.193.121:0/1419093118 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fbb90004290 con 0x7fbb9c104940 2026-02-13T10:37:58.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.726+0000 7fbb7bfff640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbb6c0772b0 0x7fbb6c079770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.726+0000 7fbb7bfff640 1 -- 10.20.193.121:0/1419093118 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fbb90098250 con 0x7fbb9c104940 2026-02-13T10:37:58.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.726+0000 7fbb9a7fc640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbb6c0772b0 0x7fbb6c079770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.726+0000 7fbb9a7fc640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbb6c0772b0 0x7fbb6c079770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb840097c0 tx=0x7fbb84005c50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.728+0000 7fbb7bfff640 1 -- 10.20.193.121:0/1419093118 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbb90066400 con 0x7fbb9c104940 2026-02-13T10:37:58.767 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.768 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.779 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 -- 10.20.193.121:0/870583772 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 msgr2=0x7f0a2c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 --2- 10.20.193.121:0/870583772 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c102c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0099b0 tx=0x7f0a1c02f220 comp rx=0 tx=0).stop 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 -- 10.20.193.121:0/870583772 shutdown_connections 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 --2- 10.20.193.121:0/870583772 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c102c00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 --2- 10.20.193.121:0/870583772 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 0x7f0a2c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 -- 10.20.193.121:0/870583772 >> 10.20.193.121:0/870583772 conn(0x7f0a2c0fcd30 msgr2=0x7f0a2c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 -- 10.20.193.121:0/870583772 shutdown_connections 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.776+0000 7f0a31c88640 1 -- 10.20.193.121:0/870583772 wait complete. 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a31c88640 1 Processor -- start 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a31c88640 1 -- start start 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a31c88640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 0x7f0a2c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.780 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a31c88640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a31c88640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10a270 con 0x7f0a2c102780 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a31c88640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10a0f0 con 0x7f0a2c101580 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c06d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57074/0 (socket says 10.20.193.121:57074) 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2affd640 1 -- 10.20.193.121:0/2637472435 learned_addr learned my addr 10.20.193.121:0/2637472435 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2b7fe640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 0x7f0a2c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2b7fe640 1 -- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 msgr2=0x7f0a2c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2b7fe640 1 -- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 msgr2=0x7f0a2c06cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2b7fe640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 0x7f0a2c06cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T10:37:58.781 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2b7fe640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 0x7f0a2c06cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T10:37:58.782 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.783 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2affd640 1 -- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 msgr2=0x7f0a2c06cc60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2affd640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 0x7f0a2c06cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2affd640 1 -- 10.20.193.121:0/2637472435 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c06d820 con 0x7f0a2c102780 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a2affd640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c06d1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c02fd00 tx=0x7f0a1c02fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a28ff9640 1 -- 10.20.193.121:0/2637472435 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f0a1c004420 con 0x7f0a2c102780 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c072200 con 0x7f0a2c102780 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.777+0000 7f0a28ff9640 1 -- 10.20.193.121:0/2637472435 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f0a1c02fe50 con 0x7f0a2c102780 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.778+0000 7f0a28ff9640 1 -- 10.20.193.121:0/2637472435 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f0a1c041a00 con 0x7f0a2c102780 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.778+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c103d00 con 0x7f0a2c102780 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.779+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c101980 con 0x7f0a2c102780 2026-02-13T10:37:58.784 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.779+0000 7f0a28ff9640 1 -- 10.20.193.121:0/2637472435 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f0a1c041cb0 con 0x7f0a2c102780 2026-02-13T10:37:58.785 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.780+0000 7f0a28ff9640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0a000772b0 0x7f0a00079770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.785 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.780+0000 7f0a28ff9640 1 -- 10.20.193.121:0/2637472435 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f0a1c03e070 con 0x7f0a2c102780 2026-02-13T10:37:58.785 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.780+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f75f4195e70 con 0x7f75f4105b40 2026-02-13T10:37:58.785 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.781+0000 7f0a2b7fe640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0a000772b0 0x7f0a00079770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.785 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.781+0000 7f0a2b7fe640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0a000772b0 0x7f0a00079770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0a18005fd0 tx=0x7f0a18005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.786 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.786 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.782+0000 7f0a28ff9640 1 -- 10.20.193.121:0/2637472435 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0a1c08ad70 con 0x7f0a2c102780 2026-02-13T10:37:58.787 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.788 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.787+0000 7f75e9ffb640 1 -- 10.20.193.121:0/3671126764 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v18)=0 v18) ==== 106+0+0 (secure 0 0 0) 0x7f75dc06bf80 con 0x7f75f4105b40 2026-02-13T10:37:58.793 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.791+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f75d0077510 msgr2=0x7f75d00799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.793 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.791+0000 7f75fab17640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f75d0077510 0x7f75d00799d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75e4002c50 tx=0x7f75e4009680 comp rx=0 tx=0).stop 2026-02-13T10:37:58.793 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.791+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4105b40 msgr2=0x7f75f419ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.793 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.791+0000 7f75fab17640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4105b40 0x7f75f419ca90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b4f0 tx=0x7f75dc00b9c0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.793 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.792+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 shutdown_connections 2026-02-13T10:37:58.793 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.792+0000 7f75fab17640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f75d0077510 0x7f75d00799d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.794 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.792+0000 7f75fab17640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f75f4105b40 0x7f75f419ca90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.794 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.792+0000 7f75fab17640 1 --2- 10.20.193.121:0/3671126764 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f75f4104940 0x7f75f419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.794 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.792+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 >> 10.20.193.121:0/3671126764 conn(0x7f75f41000f0 msgr2=0x7f75f4109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.794 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.792+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 shutdown_connections 2026-02-13T10:37:58.794 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.792+0000 7f75fab17640 1 -- 10.20.193.121:0/3671126764 wait complete. 2026-02-13T10:37:58.834 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.833+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fbb9c0786b0 con 0x7fbb9c104940 2026-02-13T10:37:58.840 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.838+0000 7fbb7bfff640 1 -- 10.20.193.121:0/1419093118 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v19)=0 v19) ==== 122+0+0 (secure 0 0 0) 0x7fbb9c0786b0 con 0x7fbb9c104940 2026-02-13T10:37:58.842 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbb6c0772b0 msgr2=0x7fbb6c079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbb6c0772b0 0x7fbb6c079770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb840097c0 tx=0x7fbb84005c50 comp rx=0 tx=0).stop 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c104940 msgr2=0x7fbb9c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c104940 0x7fbb9c197f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000d990 tx=0x7fbb9000de60 comp rx=0 tx=0).stop 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 shutdown_connections 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbb6c0772b0 0x7fbb6c079770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbb9c105b40 0x7fbb9c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 --2- 10.20.193.121:0/1419093118 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbb9c104940 0x7fbb9c197f70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 >> 10.20.193.121:0/1419093118 conn(0x7fbb9c1000f0 msgr2=0x7fbb9c109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 shutdown_connections 2026-02-13T10:37:58.843 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.841+0000 7fbba1740640 1 -- 10.20.193.121:0/1419093118 wait complete. 2026-02-13T10:37:58.880 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.878+0000 7fa4fb043640 1 -- 10.20.193.121:0/765286127 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 msgr2=0x7fa4f4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.880 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.878+0000 7fa4fb043640 1 --2- 10.20.193.121:0/765286127 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f4105fb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0099b0 tx=0x7fa4dc02f220 comp rx=0 tx=0).stop 2026-02-13T10:37:58.880 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.878+0000 7fa4fb043640 1 -- 10.20.193.121:0/765286127 shutdown_connections 2026-02-13T10:37:58.880 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.878+0000 7fa4fb043640 1 --2- 10.20.193.121:0/765286127 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f4105fb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.880 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.878+0000 7fa4fb043640 1 --2- 10.20.193.121:0/765286127 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa4f4104930 0x7fa4f4104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.881 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.878+0000 7fa4fb043640 1 -- 10.20.193.121:0/765286127 >> 10.20.193.121:0/765286127 conn(0x7fa4f41000e0 msgr2=0x7fa4f4102500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.879+0000 7fa4fb043640 1 -- 10.20.193.121:0/765286127 shutdown_connections 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.879+0000 7fa4fb043640 1 -- 10.20.193.121:0/765286127 wait complete. 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4fb043640 1 Processor -- start 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4fb043640 1 -- start start 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4fb043640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa4f4104930 0x7fa4f419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4fb043640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4fb043640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f410d620 con 0x7fa4f4105b30 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4fb043640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f410d4a0 con 0x7fa4f4104930 2026-02-13T10:37:58.883 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4ebfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4ebfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57096/0 (socket says 10.20.193.121:57096) 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4ebfff640 1 -- 10.20.193.121:0/1686198807 learned_addr learned my addr 10.20.193.121:0/1686198807 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4f8db8640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa4f4104930 0x7fa4f419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4ebfff640 1 -- 10.20.193.121:0/1686198807 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa4f4104930 msgr2=0x7fa4f419c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4ebfff640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa4f4104930 0x7fa4f419c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.880+0000 7fa4ebfff640 1 -- 10.20.193.121:0/1686198807 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f419cfb0 con 0x7fa4f4105b30 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.881+0000 7fa4ebfff640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f419c930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc02fd00 tx=0x7fa4dc02fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.881+0000 7fa4e9ffb640 1 -- 10.20.193.121:0/1686198807 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fa4dc004420 con 0x7fa4f4105b30 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.881+0000 7fa4e9ffb640 1 -- 10.20.193.121:0/1686198807 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fa4dc02fe50 con 0x7fa4f4105b30 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.881+0000 7fa4e9ffb640 1 -- 10.20.193.121:0/1686198807 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fa4dc041a00 con 0x7fa4f4105b30 2026-02-13T10:37:58.884 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.881+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f41a1990 con 0x7fa4f4105b30 2026-02-13T10:37:58.885 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.882+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f41a1fa0 con 0x7fa4f4105b30 2026-02-13T10:37:58.886 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.883+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4104d30 con 0x7fa4f4105b30 2026-02-13T10:37:58.886 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.883+0000 7fa4e9ffb640 1 -- 10.20.193.121:0/1686198807 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fa4dc041ba0 con 0x7fa4f4105b30 2026-02-13T10:37:58.886 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.884+0000 7fa4e9ffb640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa4d0077510 0x7fa4d00799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.886 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.884+0000 7fa4e9ffb640 1 -- 10.20.193.121:0/1686198807 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fa4dc03e070 con 0x7fa4f4105b30 2026-02-13T10:37:58.886 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.884+0000 7fa4f8db8640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa4d0077510 0x7fa4d00799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.887 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.884+0000 7fa4f8db8640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa4d0077510 0x7fa4d00799d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40059c0 tx=0x7fa4e4005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.889 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.887+0000 7fa4e9ffb640 1 -- 10.20.193.121:0/1686198807 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa4dc08b620 con 0x7fa4f4105b30 2026-02-13T10:37:58.892 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.891+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f0a2c1041a0 con 0x7f0a2c102780 2026-02-13T10:37:58.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.893+0000 7f0a28ff9640 1 -- 10.20.193.121:0/2637472435 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v20)=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f0a1c08fc20 con 0x7f0a2c102780 2026-02-13T10:37:58.897 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0a000772b0 msgr2=0x7f0a00079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0a000772b0 0x7f0a00079770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0a18005fd0 tx=0x7f0a18005950 comp rx=0 tx=0).stop 2026-02-13T10:37:58.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 msgr2=0x7f0a2c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c06d1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c02fd00 tx=0x7f0a1c02fd30 comp rx=0 tx=0).stop 2026-02-13T10:37:58.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 shutdown_connections 2026-02-13T10:37:58.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0a000772b0 0x7f0a00079770 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0a2c102780 0x7f0a2c06d1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.899 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 --2- 10.20.193.121:0/2637472435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0a2c101580 0x7f0a2c06cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.899 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 >> 10.20.193.121:0/2637472435 conn(0x7f0a2c0fcd30 msgr2=0x7f0a2c06aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.899 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 shutdown_connections 2026-02-13T10:37:58.899 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.896+0000 7f0a31c88640 1 -- 10.20.193.121:0/2637472435 wait complete. 2026-02-13T10:37:58.920 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 -- 10.20.193.121:0/749190894 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c104940 msgr2=0x7f424c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.920 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 --2- 10.20.193.121:0/749190894 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c104940 0x7f424c104d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42380099b0 tx=0x7f423802f220 comp rx=0 tx=0).stop 2026-02-13T10:37:58.920 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 -- 10.20.193.121:0/749190894 shutdown_connections 2026-02-13T10:37:58.920 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 --2- 10.20.193.121:0/749190894 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c105b40 0x7f424c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.920 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 --2- 10.20.193.121:0/749190894 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c104940 0x7f424c104d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.920 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 -- 10.20.193.121:0/749190894 >> 10.20.193.121:0/749190894 conn(0x7f424c1000f0 msgr2=0x7f424c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.920 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 -- 10.20.193.121:0/749190894 shutdown_connections 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 -- 10.20.193.121:0/749190894 wait complete. 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.918+0000 7f4251fde640 1 Processor -- start 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4251fde640 1 -- start start 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4251fde640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 0x7f424c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4251fde640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c105b40 0x7f424c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4251fde640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10d610 con 0x7f424c105b40 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4251fde640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10d490 con 0x7f424c104940 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c105b40 0x7f424c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c105b40 0x7f424c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57126/0 (socket says 10.20.193.121:57126) 2026-02-13T10:37:58.922 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424affd640 1 -- 10.20.193.121:0/2867415402 learned_addr learned my addr 10.20.193.121:0/2867415402 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424b7fe640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 0x7f424c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424b7fe640 1 -- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 msgr2=0x7f424c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424b7fe640 1 -- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 msgr2=0x7f424c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424b7fe640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 0x7f424c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424b7fe640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 0x7f424c197ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424affd640 1 -- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 msgr2=0x7f424c197ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424affd640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 0x7f424c197ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424affd640 1 -- 10.20.193.121:0/2867415402 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c198940 con 0x7f424c105b40 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f424affd640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c105b40 0x7f424c198400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f423c00d8d0 tx=0x7f423c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4248ff9640 1 -- 10.20.193.121:0/2867415402 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f423c014070 con 0x7f424c105b40 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c1aea70 con 0x7f424c105b40 2026-02-13T10:37:58.923 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4248ff9640 1 -- 10.20.193.121:0/2867415402 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f423c0044a0 con 0x7f424c105b40 2026-02-13T10:37:58.924 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c1aeee0 con 0x7f424c105b40 2026-02-13T10:37:58.924 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.919+0000 7f4248ff9640 1 -- 10.20.193.121:0/2867415402 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f423c01e060 con 0x7f424c105b40 2026-02-13T10:37:58.924 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.920+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424c10ddf0 con 0x7f424c105b40 2026-02-13T10:37:58.925 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.921+0000 7f4248ff9640 1 -- 10.20.193.121:0/2867415402 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f423c00b840 con 0x7f424c105b40 2026-02-13T10:37:58.925 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.921+0000 7f4248ff9640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f42200774c0 0x7f4220079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.925 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.921+0000 7f4248ff9640 1 -- 10.20.193.121:0/2867415402 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f423c098d00 con 0x7f424c105b40 2026-02-13T10:37:58.925 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.923+0000 7f424b7fe640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f42200774c0 0x7f4220079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.925 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.923+0000 7f424b7fe640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f42200774c0 0x7f4220079980 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4238005e90 tx=0x7f4238005c70 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.925 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.923+0000 7f4248ff9640 1 -- 10.20.193.121:0/2867415402 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f423c066eb0 con 0x7f424c105b40 2026-02-13T10:37:58.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.929+0000 7fe0592d4640 1 -- 10.20.193.121:0/3784652345 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054105b40 msgr2=0x7fe054105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.929+0000 7fe0592d4640 1 --2- 10.20.193.121:0/3784652345 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054105b40 0x7fe054105fc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0099b0 tx=0x7fe03c02f220 comp rx=0 tx=0).stop 2026-02-13T10:37:58.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 -- 10.20.193.121:0/3784652345 shutdown_connections 2026-02-13T10:37:58.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 --2- 10.20.193.121:0/3784652345 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054105b40 0x7fe054105fc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.932 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 --2- 10.20.193.121:0/3784652345 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe054104940 0x7fe054104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.932 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 -- 10.20.193.121:0/3784652345 >> 10.20.193.121:0/3784652345 conn(0x7fe0541000f0 msgr2=0x7fe054102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 -- 10.20.193.121:0/3784652345 shutdown_connections 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 -- 10.20.193.121:0/3784652345 wait complete. 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 Processor -- start 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 -- start start 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054104940 0x7fe05419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe054105b40 0x7fe05419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe05410d630 con 0x7fe054104940 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0592d4640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe05410d4b0 con 0x7fe054105b40 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe052ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054104940 0x7fe05419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.934 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe052ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054104940 0x7fe05419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57144/0 (socket says 10.20.193.121:57144) 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe052ffd640 1 -- 10.20.193.121:0/442023471 learned_addr learned my addr 10.20.193.121:0/442023471 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe0527fc640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe054105b40 0x7fe05419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe052ffd640 1 -- 10.20.193.121:0/442023471 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe054105b40 msgr2=0x7fe05419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe052ffd640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe054105b40 0x7fe05419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.930+0000 7fe052ffd640 1 -- 10.20.193.121:0/442023471 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05419cf80 con 0x7fe054104940 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.931+0000 7fe052ffd640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054104940 0x7fe05419c3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe04800d8d0 tx=0x7fe04800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.931+0000 7fe033fff640 1 -- 10.20.193.121:0/442023471 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fe048014070 con 0x7fe054104940 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.931+0000 7fe033fff640 1 -- 10.20.193.121:0/442023471 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fe048007e50 con 0x7fe054104940 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.931+0000 7fe033fff640 1 -- 10.20.193.121:0/442023471 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fe048005020 con 0x7fe054104940 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.931+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0541a19c0 con 0x7fe054104940 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.931+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0541a1fd0 con 0x7fe054104940 2026-02-13T10:37:58.935 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.931+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe054104d40 con 0x7fe054104940 2026-02-13T10:37:58.936 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.932+0000 7fe033fff640 1 -- 10.20.193.121:0/442023471 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fe048005290 con 0x7fe054104940 2026-02-13T10:37:58.936 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.932+0000 7fe033fff640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0280774c0 0x7fe028079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.936 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.932+0000 7fe033fff640 1 -- 10.20.193.121:0/442023471 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fe048098660 con 0x7fe054104940 2026-02-13T10:37:58.936 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.932+0000 7fe0527fc640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0280774c0 0x7fe028079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.936 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.933+0000 7fe0527fc640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0280774c0 0x7fe028079980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe05419d870 tx=0x7fe03c03a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.936 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.935+0000 7fe033fff640 1 -- 10.20.193.121:0/442023471 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe048066770 con 0x7fe054104940 2026-02-13T10:37:58.958 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.958 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.963 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.964 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:58.966 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 -- 10.20.193.121:0/3072427941 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 msgr2=0x7f9298104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3072427941 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f9298104d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f92880099b0 tx=0x7f928802f220 comp rx=0 tx=0).stop 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 -- 10.20.193.121:0/3072427941 shutdown_connections 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3072427941 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 0x7f9298105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3072427941 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f9298104d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 -- 10.20.193.121:0/3072427941 >> 10.20.193.121:0/3072427941 conn(0x7f92981000f0 msgr2=0x7f9298102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 -- 10.20.193.121:0/3072427941 shutdown_connections 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.965+0000 7f929f7f6640 1 -- 10.20.193.121:0/3072427941 wait complete. 2026-02-13T10:37:58.969 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929f7f6640 1 Processor -- start 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929f7f6640 1 -- start start 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929f7f6640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f929819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929f7f6640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 0x7f929819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929f7f6640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f929810d630 con 0x7f9298104940 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929f7f6640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f929810d4b0 con 0x7f9298105b40 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929d56b640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f929819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929d56b640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f929819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57164/0 (socket says 10.20.193.121:57164) 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929d56b640 1 -- 10.20.193.121:0/3702850914 learned_addr learned my addr 10.20.193.121:0/3702850914 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929cd6a640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 0x7f929819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929cd6a640 1 -- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 msgr2=0x7f929819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2026-02-13T10:37:58.970 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929cd6a640 1 -- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 msgr2=0x7f929819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929cd6a640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 0x7f929819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929cd6a640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 0x7f929819c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929d56b640 1 -- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 msgr2=0x7f929819c900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929d56b640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 0x7f929819c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929d56b640 1 -- 10.20.193.121:0/3702850914 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929819cf80 con 0x7f9298104940 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.966+0000 7f929d56b640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f929819c3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f928802f730 tx=0x7f928802fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.967+0000 7f92867fc640 1 -- 10.20.193.121:0/3702850914 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9288004260 con 0x7f9298104940 2026-02-13T10:37:58.971 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.967+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92981a1960 con 0x7f9298104940 2026-02-13T10:37:58.972 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.967+0000 7f92867fc640 1 -- 10.20.193.121:0/3702850914 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f9288004400 con 0x7f9298104940 2026-02-13T10:37:58.972 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.967+0000 7f92867fc640 1 -- 10.20.193.121:0/3702850914 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9288004260 con 0x7f9298104940 2026-02-13T10:37:58.973 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.967+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92981a1f70 con 0x7f9298104940 2026-02-13T10:37:58.973 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.968+0000 7f92867fc640 1 -- 10.20.193.121:0/3702850914 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f9288041910 con 0x7f9298104940 2026-02-13T10:37:58.973 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.968+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9298104d40 con 0x7f9298104940 2026-02-13T10:37:58.973 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.969+0000 7f92867fc640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f92740778c0 0x7f9274079d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:58.973 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.969+0000 7f92867fc640 1 -- 10.20.193.121:0/3702850914 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f928803e070 con 0x7f9298104940 2026-02-13T10:37:58.973 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.969+0000 7f929cd6a640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f92740778c0 0x7f9274079d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:58.973 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.969+0000 7f929cd6a640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f92740778c0 0x7f9274079d80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f929819d870 tx=0x7f928c004340 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:58.975 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.974+0000 7f92867fc640 1 -- 10.20.193.121:0/3702850914 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f928808bed0 con 0x7f9298104940 2026-02-13T10:37:58.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='trial121') 2026-02-13T10:37:58.989 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.988+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fa4f4063880 con 0x7fa4f4105b30 2026-02-13T10:37:58.992 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.990+0000 7fa4e9ffb640 1 -- 10.20.193.121:0/1686198807 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fa4dc0904d0 con 0x7fa4f4105b30 2026-02-13T10:37:58.994 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa4d0077510 msgr2=0x7fa4d00799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa4d0077510 0x7fa4d00799d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40059c0 tx=0x7fa4e4005950 comp rx=0 tx=0).stop 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 msgr2=0x7fa4f419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f419c930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc02fd00 tx=0x7fa4dc02fd30 comp rx=0 tx=0).stop 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 shutdown_connections 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa4d0077510 0x7fa4d00799d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa4f4105b30 0x7fa4f419c930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 --2- 10.20.193.121:0/1686198807 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa4f4104930 0x7fa4f419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 >> 10.20.193.121:0/1686198807 conn(0x7fa4f41000e0 msgr2=0x7fa4f4109180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 shutdown_connections 2026-02-13T10:37:58.995 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:58.993+0000 7fa4fb043640 1 -- 10.20.193.121:0/1686198807 wait complete. 2026-02-13T10:37:59.026 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.024+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f424c078c30 con 0x7f424c105b40 2026-02-13T10:37:59.029 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.027+0000 7f4248ff9640 1 -- 10.20.193.121:0/2867415402 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v22)=0 v22) ==== 119+0+0 (secure 0 0 0) 0x7f423c06bd60 con 0x7f424c105b40 2026-02-13T10:37:59.032 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:59.032 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:59.033 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f42200774c0 msgr2=0x7f4220079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.033 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f42200774c0 0x7f4220079980 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4238005e90 tx=0x7f4238005c70 comp rx=0 tx=0).stop 2026-02-13T10:37:59.033 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c105b40 msgr2=0x7f424c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c105b40 0x7f424c198400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f423c00d8d0 tx=0x7f423c00dda0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 shutdown_connections 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f42200774c0 0x7f4220079980 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f424c105b40 0x7f424c198400 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 --2- 10.20.193.121:0/2867415402 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f424c104940 0x7f424c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 >> 10.20.193.121:0/2867415402 conn(0x7f424c1000f0 msgr2=0x7f424c109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 shutdown_connections 2026-02-13T10:37:59.034 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.032+0000 7f4251fde640 1 -- 10.20.193.121:0/2867415402 wait complete. 2026-02-13T10:37:59.037 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.036+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fe054195df0 con 0x7fe054104940 2026-02-13T10:37:59.042 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.040+0000 7fe033fff640 1 -- 10.20.193.121:0/442023471 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fe04806b620 con 0x7fe054104940 2026-02-13T10:37:59.045 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0280774c0 msgr2=0x7fe028079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.045 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0280774c0 0x7fe028079980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe05419d870 tx=0x7fe03c03a040 comp rx=0 tx=0).stop 2026-02-13T10:37:59.045 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054104940 msgr2=0x7fe05419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.045 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054104940 0x7fe05419c3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe04800d8d0 tx=0x7fe04800dda0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 shutdown_connections 2026-02-13T10:37:59.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0280774c0 0x7fe028079980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe054105b40 0x7fe05419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 --2- 10.20.193.121:0/442023471 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe054104940 0x7fe05419c3c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.043+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 >> 10.20.193.121:0/442023471 conn(0x7fe0541000f0 msgr2=0x7fe054109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.044+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 shutdown_connections 2026-02-13T10:37:59.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.044+0000 7fe0592d4640 1 -- 10.20.193.121:0/442023471 wait complete. 2026-02-13T10:37:59.047 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:59.051 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 -- 10.20.193.121:0/2812895132 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 msgr2=0x7fae3c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 --2- 10.20.193.121:0/2812895132 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c105fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0099b0 tx=0x7fae2c02f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 -- 10.20.193.121:0/2812895132 shutdown_connections 2026-02-13T10:37:59.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 --2- 10.20.193.121:0/2812895132 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c105fc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 --2- 10.20.193.121:0/2812895132 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae3c104940 0x7fae3c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 -- 10.20.193.121:0/2812895132 >> 10.20.193.121:0/2812895132 conn(0x7fae3c1000f0 msgr2=0x7fae3c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 -- 10.20.193.121:0/2812895132 shutdown_connections 2026-02-13T10:37:59.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.068+0000 7fae41f1b640 1 -- 10.20.193.121:0/2812895132 wait complete. 2026-02-13T10:37:59.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 Processor -- start 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 -- start start 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae3c104940 0x7fae3c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c10d630 con 0x7fae3c105b40 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c10d4b0 con 0x7fae3c104940 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57172/0 (socket says 10.20.193.121:57172) 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3affd640 1 -- 10.20.193.121:0/3158643384 learned_addr learned my addr 10.20.193.121:0/3158643384 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3b7fe640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae3c104940 0x7fae3c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.072 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3affd640 1 -- 10.20.193.121:0/3158643384 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae3c104940 msgr2=0x7fae3c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3affd640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae3c104940 0x7fae3c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3affd640 1 -- 10.20.193.121:0/3158643384 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3c198ab0 con 0x7fae3c105b40 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae3affd640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c1983e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fae2c02fd00 tx=0x7fae2c02fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae38ff9640 1 -- 10.20.193.121:0/3158643384 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fae2c004420 con 0x7fae3c105b40 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3c19d480 con 0x7fae3c105b40 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae38ff9640 1 -- 10.20.193.121:0/3158643384 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fae2c02fe50 con 0x7fae3c105b40 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae38ff9640 1 -- 10.20.193.121:0/3158643384 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fae2c041a00 con 0x7fae3c105b40 2026-02-13T10:37:59.073 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.069+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3c19da30 con 0x7fae3c105b40 2026-02-13T10:37:59.074 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.070+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3c104d40 con 0x7fae3c105b40 2026-02-13T10:37:59.074 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.071+0000 7fae38ff9640 1 -- 10.20.193.121:0/3158643384 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fae2c041ba0 con 0x7fae3c105b40 2026-02-13T10:37:59.074 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.071+0000 7fae38ff9640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae10077560 0x7fae10079a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.074 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.071+0000 7fae38ff9640 1 -- 10.20.193.121:0/3158643384 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fae2c03e070 con 0x7fae3c105b40 2026-02-13T10:37:59.074 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.071+0000 7fae3b7fe640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae10077560 0x7fae10079a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.074 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.071+0000 7fae3b7fe640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae10077560 0x7fae10079a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae280059c0 tx=0x7fae28005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.075 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.073+0000 7fae38ff9640 1 -- 10.20.193.121:0/3158643384 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fae2c049340 con 0x7fae3c105b40 2026-02-13T10:37:59.082 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 -- 10.20.193.121:0/1194853568 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074104940 msgr2=0x7fb074104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.082 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 --2- 10.20.193.121:0/1194853568 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074104940 0x7fb074104d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb0640099b0 tx=0x7fb06402f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.083 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 -- 10.20.193.121:0/1194853568 shutdown_connections 2026-02-13T10:37:59.083 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 --2- 10.20.193.121:0/1194853568 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb074105b40 0x7fb074105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.083 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 --2- 10.20.193.121:0/1194853568 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074104940 0x7fb074104d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.083 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 -- 10.20.193.121:0/1194853568 >> 10.20.193.121:0/1194853568 conn(0x7fb0741000f0 msgr2=0x7fb074102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.083 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 -- 10.20.193.121:0/1194853568 shutdown_connections 2026-02-13T10:37:59.083 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7fb07c57f640 1 -- 10.20.193.121:0/1194853568 wait complete. 2026-02-13T10:37:59.083 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.080+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f9298195fc0 con 0x7f9298104940 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07c57f640 1 Processor -- start 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07c57f640 1 -- start start 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07c57f640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb074104940 0x7fb07419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07c57f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074105b40 0x7fb07419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07c57f640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb07410d630 con 0x7fb074105b40 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07c57f640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb07410d4b0 con 0x7fb074104940 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb079af3640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074105b40 0x7fb07419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb079af3640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074105b40 0x7fb07419c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57182/0 (socket says 10.20.193.121:57182) 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb079af3640 1 -- 10.20.193.121:0/2056664596 learned_addr learned my addr 10.20.193.121:0/2056664596 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.086 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07a2f4640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb074104940 0x7fb07419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb079af3640 1 -- 10.20.193.121:0/2056664596 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb074104940 msgr2=0x7fb07419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb079af3640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb074104940 0x7fb07419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb079af3640 1 -- 10.20.193.121:0/2056664596 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07419d060 con 0x7fb074105b40 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.081+0000 7fb07a2f4640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb074104940 0x7fb07419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.082+0000 7fb079af3640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074105b40 0x7fb07419c9e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d900 tx=0x7fb06800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.082+0000 7fb05b7fe640 1 -- 10.20.193.121:0/2056664596 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fb068014070 con 0x7fb074105b40 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.082+0000 7fb05b7fe640 1 -- 10.20.193.121:0/2056664596 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fb0680044a0 con 0x7fb074105b40 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.082+0000 7fb05b7fe640 1 -- 10.20.193.121:0/2056664596 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fb068005020 con 0x7fb074105b40 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.082+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0741a1aa0 con 0x7fb074105b40 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.082+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0741a2050 con 0x7fb074105b40 2026-02-13T10:37:59.087 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.083+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb074104d40 con 0x7fb074105b40 2026-02-13T10:37:59.088 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.085+0000 7f92867fc640 1 -- 10.20.193.121:0/3702850914 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v24)=0 v24) ==== 120+0+0 (secure 0 0 0) 0x7f9288090d80 con 0x7f9298104940 2026-02-13T10:37:59.088 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.087+0000 7fb05b7fe640 1 -- 10.20.193.121:0/2056664596 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fb0680051c0 con 0x7fb074105b40 2026-02-13T10:37:59.089 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.087+0000 7fb05b7fe640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb050077560 0x7fb050079a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.089 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.087+0000 7fb05b7fe640 1 -- 10.20.193.121:0/2056664596 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fb068098940 con 0x7fb074105b40 2026-02-13T10:37:59.089 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.088+0000 7fb07a2f4640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb050077560 0x7fb050079a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.090 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.088+0000 7fb07a2f4640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb050077560 0x7fb050079a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0640097c0 tx=0x7fb0640023d0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.090 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.088+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f92740778c0 msgr2=0x7f9274079d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.090 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.088+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f92740778c0 0x7f9274079d80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f929819d870 tx=0x7f928c004340 comp rx=0 tx=0).stop 2026-02-13T10:37:59.090 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.088+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 msgr2=0x7f929819c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.090 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.088+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f929819c3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f928802f730 tx=0x7f928802fd00 comp rx=0 tx=0).stop 2026-02-13T10:37:59.091 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7fb05b7fe640 1 -- 10.20.193.121:0/2056664596 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb068066b50 con 0x7fb074105b40 2026-02-13T10:37:59.091 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 shutdown_connections 2026-02-13T10:37:59.091 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f92740778c0 0x7f9274079d80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.091 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9298105b40 0x7f929819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.091 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f929f7f6640 1 --2- 10.20.193.121:0/3702850914 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9298104940 0x7f929819c3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.092 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 >> 10.20.193.121:0/3702850914 conn(0x7f92981000f0 msgr2=0x7f9298109140 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.092 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 -- 10.20.193.121:0/978838020 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 msgr2=0x7f712c075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.092 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 --2- 10.20.193.121:0/978838020 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c075500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71180099b0 tx=0x7f711802f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.092 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 -- 10.20.193.121:0/978838020 shutdown_connections 2026-02-13T10:37:59.093 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 --2- 10.20.193.121:0/978838020 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f712c075a40 0x7f712c075ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.093 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 --2- 10.20.193.121:0/978838020 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c075500 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.093 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 -- 10.20.193.121:0/978838020 >> 10.20.193.121:0/978838020 conn(0x7f712c0fde60 msgr2=0x7f712c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.093 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 -- 10.20.193.121:0/978838020 shutdown_connections 2026-02-13T10:37:59.093 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.089+0000 7f7131afb640 1 -- 10.20.193.121:0/978838020 wait complete. 2026-02-13T10:37:59.093 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 shutdown_connections 2026-02-13T10:37:59.093 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f929f7f6640 1 -- 10.20.193.121:0/3702850914 wait complete. 2026-02-13T10:37:59.095 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f7131afb640 1 Processor -- start 2026-02-13T10:37:59.095 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f7131afb640 1 -- start start 2026-02-13T10:37:59.095 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f7131afb640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f712c075a40 0x7f712c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.095 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f7131afb640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.095 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f7131afb640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f712c111af0 con 0x7f712c0770a0 2026-02-13T10:37:59.095 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f7131afb640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f712c111970 con 0x7f712c075a40 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c1a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57196/0 (socket says 10.20.193.121:57196) 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712affd640 1 -- 10.20.193.121:0/4225469871 learned_addr learned my addr 10.20.193.121:0/4225469871 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712b7fe640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f712c075a40 0x7f712c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712affd640 1 -- 10.20.193.121:0/4225469871 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f712c075a40 msgr2=0x7f712c1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712affd640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f712c075a40 0x7f712c1a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712affd640 1 -- 10.20.193.121:0/4225469871 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c1a14d0 con 0x7f712c0770a0 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.090+0000 7f712b7fe640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f712c075a40 0x7f712c1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.091+0000 7f712affd640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c1a0e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d8d0 tx=0x7f711c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.091+0000 7f7128ff9640 1 -- 10.20.193.121:0/4225469871 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f711c014070 con 0x7f712c0770a0 2026-02-13T10:37:59.096 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.091+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c1a5f10 con 0x7f712c0770a0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.091+0000 7f7128ff9640 1 -- 10.20.193.121:0/4225469871 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f711c0044a0 con 0x7f712c0770a0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.091+0000 7f7128ff9640 1 -- 10.20.193.121:0/4225469871 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f711c002e60 con 0x7f712c0770a0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.091+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1a6490 con 0x7f712c0770a0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.092+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c075ea0 con 0x7f712c0770a0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.092+0000 7f7128ff9640 1 -- 10.20.193.121:0/4225469871 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f711c00b840 con 0x7f712c0770a0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.093+0000 7f7128ff9640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7100077510 0x7f71000799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.093+0000 7f7128ff9640 1 -- 10.20.193.121:0/4225469871 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f711c0988a0 con 0x7f712c0770a0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.093+0000 7f712b7fe640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7100077510 0x7f71000799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.097 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.094+0000 7f712b7fe640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7100077510 0x7f71000799d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f71180097c0 tx=0x7f71180023d0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.098 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f7128ff9640 1 -- 10.20.193.121:0/4225469871 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f711c066a30 con 0x7f712c0770a0 2026-02-13T10:37:59.098 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f86fb024640 1 -- 10.20.193.121:0/241059309 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4103190 msgr2=0x7f86f4105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.098 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f86fb024640 1 --2- 10.20.193.121:0/241059309 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4103190 0x7f86f4105580 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f86e40099b0 tx=0x7f86e402f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.099 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f4f19030640 1 -- 10.20.193.121:0/1258137384 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14101590 msgr2=0x7f4f14101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.099 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f4f19030640 1 --2- 10.20.193.121:0/1258137384 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14101590 0x7f4f14101990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f080099b0 tx=0x7f4f0802f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.099 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 -- 10.20.193.121:0/1258137384 shutdown_connections 2026-02-13T10:37:59.100 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 --2- 10.20.193.121:0/1258137384 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4f14102790 0x7f4f14102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.100 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 --2- 10.20.193.121:0/1258137384 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14101590 0x7f4f14101990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.100 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 -- 10.20.193.121:0/1258137384 >> 10.20.193.121:0/1258137384 conn(0x7f4f140fcd20 msgr2=0x7f4f140ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.100 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 -- 10.20.193.121:0/1258137384 shutdown_connections 2026-02-13T10:37:59.100 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 -- 10.20.193.121:0/1258137384 wait complete. 2026-02-13T10:37:59.100 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f86fb024640 1 -- 10.20.193.121:0/241059309 shutdown_connections 2026-02-13T10:37:59.100 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f86fb024640 1 --2- 10.20.193.121:0/241059309 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f86f4105ac0 0x7f86f4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.101 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f86fb024640 1 --2- 10.20.193.121:0/241059309 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4103190 0x7f86f4105580 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.101 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f86fb024640 1 -- 10.20.193.121:0/241059309 >> 10.20.193.121:0/241059309 conn(0x7f86f40fcd30 msgr2=0x7f86f40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.101 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.095+0000 7f86fb024640 1 -- 10.20.193.121:0/241059309 shutdown_connections 2026-02-13T10:37:59.101 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f86fb024640 1 -- 10.20.193.121:0/241059309 wait complete. 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 Processor -- start 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 -- start start 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4f14101590 0x7f4f141a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14102790 0x7f4f141a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1410a260 con 0x7f4f14102790 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f19030640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1410a0e0 con 0x7f4f14101590 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f12575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14102790 0x7f4f141a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f12575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14102790 0x7f4f141a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57226/0 (socket says 10.20.193.121:57226) 2026-02-13T10:37:59.102 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f4f12575640 1 -- 10.20.193.121:0/2343501013 learned_addr learned my addr 10.20.193.121:0/2343501013 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.103 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f86fb024640 1 Processor -- start 2026-02-13T10:37:59.103 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.096+0000 7f86fb024640 1 -- start start 2026-02-13T10:37:59.103 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f4f12575640 1 -- 10.20.193.121:0/2343501013 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4f14101590 msgr2=0x7f4f141a0990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T10:37:59.104 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86fb024640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f86f4103190 0x7f86f419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.104 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86fb024640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4105ac0 0x7f86f419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.104 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86fb024640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86f410b430 con 0x7f86f4105ac0 2026-02-13T10:37:59.104 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86fb024640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f86f410b2b0 con 0x7f86f4103190 2026-02-13T10:37:59.105 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f4f12575640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4f14101590 0x7f4f141a0990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.105 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f4f12575640 1 -- 10.20.193.121:0/2343501013 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f141a1550 con 0x7f4f14102790 2026-02-13T10:37:59.106 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86ebfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4105ac0 0x7f86f419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.106 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86ebfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4105ac0 0x7f86f419a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57238/0 (socket says 10.20.193.121:57238) 2026-02-13T10:37:59.106 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86ebfff640 1 -- 10.20.193.121:0/2291430474 learned_addr learned my addr 10.20.193.121:0/2291430474 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.106 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86f8d99640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f86f4103190 0x7f86f419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.106 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86ebfff640 1 -- 10.20.193.121:0/2291430474 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f86f4103190 msgr2=0x7f86f419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.106 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86ebfff640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f86f4103190 0x7f86f419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.106 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86ebfff640 1 -- 10.20.193.121:0/2291430474 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f419aee0 con 0x7f86f4105ac0 2026-02-13T10:37:59.107 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f4f12575640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14102790 0x7f4f141a0ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d8d0 tx=0x7f4efc00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.107 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86ebfff640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4105ac0 0x7f86f419a860 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00b4f0 tx=0x7f86dc00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.107 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f4ef3fff640 1 -- 10.20.193.121:0/2343501013 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4efc014070 con 0x7f4f14102790 2026-02-13T10:37:59.107 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f14195e70 con 0x7f4f14102790 2026-02-13T10:37:59.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f14196480 con 0x7f4f14102790 2026-02-13T10:37:59.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86e9ffb640 1 -- 10.20.193.121:0/2291430474 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f86dc004440 con 0x7f86f4105ac0 2026-02-13T10:37:59.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86e9ffb640 1 -- 10.20.193.121:0/2291430474 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f86dc015070 con 0x7f86f4105ac0 2026-02-13T10:37:59.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f419f8d0 con 0x7f86f4105ac0 2026-02-13T10:37:59.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86e9ffb640 1 -- 10.20.193.121:0/2291430474 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f86dc010b50 con 0x7f86f4105ac0 2026-02-13T10:37:59.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.097+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f419fee0 con 0x7f86f4105ac0 2026-02-13T10:37:59.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.098+0000 7f4ef3fff640 1 -- 10.20.193.121:0/2343501013 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f4efc0044a0 con 0x7f4f14102790 2026-02-13T10:37:59.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.098+0000 7f4ef3fff640 1 -- 10.20.193.121:0/2343501013 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4efc002e30 con 0x7f4f14102790 2026-02-13T10:37:59.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.098+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f14101990 con 0x7f4f14102790 2026-02-13T10:37:59.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.098+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f41035f0 con 0x7f86f4105ac0 2026-02-13T10:37:59.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.099+0000 7f4ef3fff640 1 -- 10.20.193.121:0/2343501013 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f4efc00b840 con 0x7f4f14102790 2026-02-13T10:37:59.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.099+0000 7f86e9ffb640 1 -- 10.20.193.121:0/2291430474 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f86dc005ac0 con 0x7f86f4105ac0 2026-02-13T10:37:59.111 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.099+0000 7f4ef3fff640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4ee8077510 0x7f4ee80799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.111 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.100+0000 7f4ef3fff640 1 -- 10.20.193.121:0/2343501013 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f4efc0988b0 con 0x7f4f14102790 2026-02-13T10:37:59.111 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.100+0000 7f4f12d76640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4ee8077510 0x7f4ee80799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.100+0000 7f86e9ffb640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f86d00774c0 0x7f86d0079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.100+0000 7f86e9ffb640 1 -- 10.20.193.121:0/2291430474 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f86dc0987c0 con 0x7f86f4105ac0 2026-02-13T10:37:59.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.100+0000 7f86f8d99640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f86d00774c0 0x7f86d0079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.100+0000 7f86f8d99640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f86d00774c0 0x7f86d0079980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86e40097c0 tx=0x7f86e403a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.101+0000 7f86e9ffb640 1 -- 10.20.193.121:0/2291430474 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f86dc010e20 con 0x7f86f4105ac0 2026-02-13T10:37:59.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.104+0000 7f4f12d76640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4ee8077510 0x7f4ee80799d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097c0 tx=0x7f4f080023d0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.104+0000 7f4ef3fff640 1 -- 10.20.193.121:0/2343501013 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4efc066ae0 con 0x7f4f14102790 2026-02-13T10:37:59.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2088319062 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec100ce0 msgr2=0x7ffbec1010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2088319062 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec100ce0 0x7ffbec1010e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00ab00 tx=0x7ffbdc030630 comp rx=0 tx=0).stop 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2088319062 shutdown_connections 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2088319062 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ffbec101ee0 0x7ffbec102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2088319062 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec100ce0 0x7ffbec1010e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2088319062 >> 10.20.193.121:0/2088319062 conn(0x7ffbec0fc470 msgr2=0x7ffbec0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2088319062 shutdown_connections 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2088319062 wait complete. 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.144+0000 7ffbf3de6640 1 Processor -- start 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf3de6640 1 -- start start 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf3de6640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ffbec100ce0 0x7ffbec104c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf3de6640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec101ee0 0x7ffbec1032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf3de6640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec109110 con 0x7ffbec101ee0 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf3de6640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec108f90 con 0x7ffbec100ce0 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf135a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec101ee0 0x7ffbec1032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf135a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec101ee0 0x7ffbec1032c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57264/0 (socket says 10.20.193.121:57264) 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf135a640 1 -- 10.20.193.121:0/2956829499 learned_addr learned my addr 10.20.193.121:0/2956829499 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf1b5b640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ffbec100ce0 0x7ffbec104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf135a640 1 -- 10.20.193.121:0/2956829499 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ffbec100ce0 msgr2=0x7ffbec104c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf135a640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ffbec100ce0 0x7ffbec104c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf135a640 1 -- 10.20.193.121:0/2956829499 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec103800 con 0x7ffbec101ee0 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf135a640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec101ee0 0x7ffbec1032c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000d8d0 tx=0x7ffbe000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbdaffd640 1 -- 10.20.193.121:0/2956829499 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ffbe0014070 con 0x7ffbec101ee0 2026-02-13T10:37:59.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbec103af0 con 0x7ffbec101ee0 2026-02-13T10:37:59.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.145+0000 7ffbdaffd640 1 -- 10.20.193.121:0/2956829499 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7ffbe00044a0 con 0x7ffbec101ee0 2026-02-13T10:37:59.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.146+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbec104040 con 0x7ffbec101ee0 2026-02-13T10:37:59.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.146+0000 7ffbdaffd640 1 -- 10.20.193.121:0/2956829499 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ffbe001e060 con 0x7ffbec101ee0 2026-02-13T10:37:59.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.146+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbec1010e0 con 0x7ffbec101ee0 2026-02-13T10:37:59.151 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.151 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.147+0000 7ffbdaffd640 1 -- 10.20.193.121:0/2956829499 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7ffbe000b840 con 0x7ffbec101ee0 2026-02-13T10:37:59.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.147+0000 7ffbdaffd640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ffbc8077560 0x7ffbc8079a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.147+0000 7ffbdaffd640 1 -- 10.20.193.121:0/2956829499 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7ffbe0098860 con 0x7ffbec101ee0 2026-02-13T10:37:59.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.147+0000 7ffbf1b5b640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ffbc8077560 0x7ffbc8079a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.148+0000 7ffbf1b5b640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ffbc8077560 0x7ffbc8079a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc005e90 tx=0x7ffbdc037670 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.150+0000 7ffbdaffd640 1 -- 10.20.193.121:0/2956829499 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ffbe0066a70 con 0x7ffbec101ee0 2026-02-13T10:37:59.155 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.173+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fae3c078900 con 0x7fae3c105b40 2026-02-13T10:37:59.177 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.176+0000 7fae38ff9640 1 -- 10.20.193.121:0/3158643384 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v25)=0 v25) ==== 119+0+0 (secure 0 0 0) 0x7fae2c08b870 con 0x7fae3c105b40 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae10077560 msgr2=0x7fae10079a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae10077560 0x7fae10079a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae280059c0 tx=0x7fae28005950 comp rx=0 tx=0).stop 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 msgr2=0x7fae3c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c1983e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fae2c02fd00 tx=0x7fae2c02fd30 comp rx=0 tx=0).stop 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 shutdown_connections 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae10077560 0x7fae10079a20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae3c105b40 0x7fae3c1983e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 --2- 10.20.193.121:0/3158643384 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae3c104940 0x7fae3c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 >> 10.20.193.121:0/3158643384 conn(0x7fae3c1000f0 msgr2=0x7fae3c109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 shutdown_connections 2026-02-13T10:37:59.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.178+0000 7fae41f1b640 1 -- 10.20.193.121:0/3158643384 wait complete. 2026-02-13T10:37:59.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- 10.20.193.121:0/3273778859 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 msgr2=0x7f6f54105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/3273778859 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f54105fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0099b0 tx=0x7f6f3c02f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- 10.20.193.121:0/3273778859 shutdown_connections 2026-02-13T10:37:59.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/3273778859 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f54105fc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/3273778859 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6f54104940 0x7f6f54104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- 10.20.193.121:0/3273778859 >> 10.20.193.121:0/3273778859 conn(0x7f6f541000f0 msgr2=0x7f6f54102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- 10.20.193.121:0/3273778859 shutdown_connections 2026-02-13T10:37:59.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- 10.20.193.121:0/3273778859 wait complete. 2026-02-13T10:37:59.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fb074195e40 con 0x7fb074105b40 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 Processor -- start 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- start start 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6f54104940 0x7f6f5407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f5407b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5410d610 con 0x7f6f54105b40 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.185+0000 7f6f5ad01640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5410d490 con 0x7f6f54104940 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.186+0000 7f6f4bfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f5407b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.186+0000 7f6f4bfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f5407b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57272/0 (socket says 10.20.193.121:57272) 2026-02-13T10:37:59.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.186+0000 7f6f4bfff640 1 -- 10.20.193.121:0/2611132715 learned_addr learned my addr 10.20.193.121:0/2611132715 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.186+0000 7f6f4bfff640 1 -- 10.20.193.121:0/2611132715 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6f54104940 msgr2=0x7f6f5407ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.186+0000 7f6f4bfff640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6f54104940 0x7f6f5407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.186+0000 7f6f4bfff640 1 -- 10.20.193.121:0/2611132715 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f54077900 con 0x7f6f54105b40 2026-02-13T10:37:59.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.186+0000 7f6f4bfff640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f5407b370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c038660 tx=0x7f6f3c002f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.188+0000 7fb05b7fe640 1 -- 10.20.193.121:0/2056664596 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v26)=0 v26) ==== 111+0+0 (secure 0 0 0) 0x7fb06806ba00 con 0x7fb074105b40 2026-02-13T10:37:59.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.189+0000 7f6f49ffb640 1 -- 10.20.193.121:0/2611132715 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6f3c046070 con 0x7f6f54105b40 2026-02-13T10:37:59.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.189+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f54077b90 con 0x7f6f54105b40 2026-02-13T10:37:59.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.189+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f54078140 con 0x7f6f54105b40 2026-02-13T10:37:59.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.189+0000 7f6f49ffb640 1 -- 10.20.193.121:0/2611132715 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f6f3c0044b0 con 0x7f6f54105b40 2026-02-13T10:37:59.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.189+0000 7f6f49ffb640 1 -- 10.20.193.121:0/2611132715 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6f3c050060 con 0x7f6f54105b40 2026-02-13T10:37:59.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.189+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f54104d40 con 0x7f6f54105b40 2026-02-13T10:37:59.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7f6f49ffb640 1 -- 10.20.193.121:0/2611132715 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f6f3c038770 con 0x7f6f54105b40 2026-02-13T10:37:59.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f712c1a6f40 con 0x7f712c0770a0 2026-02-13T10:37:59.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb050077560 msgr2=0x7fb050079a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb050077560 0x7fb050079a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0640097c0 tx=0x7fb0640023d0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074105b40 msgr2=0x7fb07419c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074105b40 0x7fb07419c9e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d900 tx=0x7fb06800ddd0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 shutdown_connections 2026-02-13T10:37:59.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb050077560 0x7fb050079a20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb074105b40 0x7fb07419c9e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 --2- 10.20.193.121:0/2056664596 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb074104940 0x7fb07419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.190+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 >> 10.20.193.121:0/2056664596 conn(0x7fb0741000f0 msgr2=0x7fb0741091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.191+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 shutdown_connections 2026-02-13T10:37:59.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.191+0000 7fb07c57f640 1 -- 10.20.193.121:0/2056664596 wait complete. 2026-02-13T10:37:59.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.191+0000 7f6f49ffb640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6f300774c0 0x7f6f30079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.191+0000 7f6f58a76640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6f300774c0 0x7f6f30079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.191+0000 7f6f49ffb640 1 -- 10.20.193.121:0/2611132715 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f6f3c03e070 con 0x7f6f54105b40 2026-02-13T10:37:59.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.191+0000 7f6f58a76640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6f300774c0 0x7f6f30079980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f541059a0 tx=0x7f6f44005d90 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.193+0000 7f7128ff9640 1 -- 10.20.193.121:0/4225469871 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v26)=0 v26) ==== 106+0+0 (secure 0 0 0) 0x7f711c06b8e0 con 0x7f712c0770a0 2026-02-13T10:37:59.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.193+0000 7f6f49ffb640 1 -- 10.20.193.121:0/2611132715 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6f3c038a90 con 0x7f6f54105b40 2026-02-13T10:37:59.197 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7100077510 msgr2=0x7f71000799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7100077510 0x7f71000799d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f71180097c0 tx=0x7f71180023d0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 msgr2=0x7f712c1a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c1a0e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d8d0 tx=0x7f711c00dda0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 shutdown_connections 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7100077510 0x7f71000799d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f712c0770a0 0x7f712c1a0e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 --2- 10.20.193.121:0/4225469871 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f712c075a40 0x7f712c1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 >> 10.20.193.121:0/4225469871 conn(0x7f712c0fde60 msgr2=0x7f712c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 shutdown_connections 2026-02-13T10:37:59.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.196+0000 7f7131afb640 1 -- 10.20.193.121:0/4225469871 wait complete. 2026-02-13T10:37:59.199 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.198+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f86f41a02f0 con 0x7f86f4105ac0 2026-02-13T10:37:59.199 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.198+0000 7f86e9ffb640 1 -- 10.20.193.121:0/2291430474 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v26)=0 v26) ==== 108+0+0 (secure 0 0 0) 0x7f86dc066950 con 0x7f86f4105ac0 2026-02-13T10:37:59.202 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.200+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f86d00774c0 msgr2=0x7f86d0079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.202 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.200+0000 7f86fb024640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f86d00774c0 0x7f86d0079980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86e40097c0 tx=0x7f86e403a040 comp rx=0 tx=0).stop 2026-02-13T10:37:59.202 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.200+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4105ac0 msgr2=0x7f86f419a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.202 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.200+0000 7f86fb024640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4105ac0 0x7f86f419a860 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00b4f0 tx=0x7f86dc00b9c0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.202 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.200+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 shutdown_connections 2026-02-13T10:37:59.203 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.200+0000 7f86fb024640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f86d00774c0 0x7f86d0079980 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.203 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.201+0000 7f86fb024640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f86f4105ac0 0x7f86f419a860 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.203 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.201+0000 7f86fb024640 1 --2- 10.20.193.121:0/2291430474 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f86f4103190 0x7f86f419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.203 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.201+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 >> 10.20.193.121:0/2291430474 conn(0x7f86f40fcd30 msgr2=0x7f86f40ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.203 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.201+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 shutdown_connections 2026-02-13T10:37:59.204 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.201+0000 7f86fb024640 1 -- 10.20.193.121:0/2291430474 wait complete. 2026-02-13T10:37:59.209 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.208+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f4f141967a0 con 0x7f4f14102790 2026-02-13T10:37:59.212 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.211+0000 7f4ef3fff640 1 -- 10.20.193.121:0/2343501013 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7f4efc06b990 con 0x7f4f14102790 2026-02-13T10:37:59.215 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4ee8077510 msgr2=0x7f4ee80799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.215 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4ee8077510 0x7f4ee80799d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097c0 tx=0x7f4f080023d0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.215 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14102790 msgr2=0x7f4f141a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.215 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14102790 0x7f4f141a0ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d8d0 tx=0x7f4efc00dda0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.215 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 shutdown_connections 2026-02-13T10:37:59.216 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4ee8077510 0x7f4ee80799d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.216 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4f14102790 0x7f4f141a0ed0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.216 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 --2- 10.20.193.121:0/2343501013 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4f14101590 0x7f4f141a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.216 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 >> 10.20.193.121:0/2343501013 conn(0x7f4f140fcd20 msgr2=0x7f4f1406aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.216 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 shutdown_connections 2026-02-13T10:37:59.216 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.213+0000 7f4f19030640 1 -- 10.20.193.121:0/2343501013 wait complete. 2026-02-13T10:37:59.251 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.249+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7ffbec1938e0 con 0x7ffbec101ee0 2026-02-13T10:37:59.253 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.252+0000 7ffbdaffd640 1 -- 10.20.193.121:0/2956829499 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7ffbe006b920 con 0x7ffbec101ee0 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ffbc8077560 msgr2=0x7ffbc8079a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ffbc8077560 0x7ffbc8079a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc005e90 tx=0x7ffbdc037670 comp rx=0 tx=0).stop 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec101ee0 msgr2=0x7ffbec1032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec101ee0 0x7ffbec1032c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000d8d0 tx=0x7ffbe000dda0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 shutdown_connections 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ffbc8077560 0x7ffbc8079a20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ffbec101ee0 0x7ffbec1032c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 --2- 10.20.193.121:0/2956829499 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ffbec100ce0 0x7ffbec104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.256 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 >> 10.20.193.121:0/2956829499 conn(0x7ffbec0fc470 msgr2=0x7ffbec1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.257 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 shutdown_connections 2026-02-13T10:37:59.257 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.254+0000 7ffbf3de6640 1 -- 10.20.193.121:0/2956829499 wait complete. 2026-02-13T10:37:59.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 -- 10.20.193.121:0/310071118 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384105ac0 msgr2=0x7fc384107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/310071118 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384105ac0 0x7fc384107eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc3780099b0 tx=0x7fc37802f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 -- 10.20.193.121:0/310071118 shutdown_connections 2026-02-13T10:37:59.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/310071118 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384105ac0 0x7fc384107eb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/310071118 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc384103190 0x7fc384105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 -- 10.20.193.121:0/310071118 >> 10.20.193.121:0/310071118 conn(0x7fc3840fcd30 msgr2=0x7fc3840ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 -- 10.20.193.121:0/310071118 shutdown_connections 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 -- 10.20.193.121:0/310071118 wait complete. 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 Processor -- start 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.257+0000 7fc38bbe9640 1 -- start start 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38bbe9640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384103190 0x7fc384102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38bbe9640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc384105ac0 0x7fc3841010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38bbe9640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc38410b3b0 con 0x7fc384103190 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38bbe9640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc38410b230 con 0x7fc384105ac0 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38995e640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384103190 0x7fc384102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38995e640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384103190 0x7fc384102a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57284/0 (socket says 10.20.193.121:57284) 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38995e640 1 -- 10.20.193.121:0/2455127011 learned_addr learned my addr 10.20.193.121:0/2455127011 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38915d640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc384105ac0 0x7fc3841010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38995e640 1 -- 10.20.193.121:0/2455127011 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc384105ac0 msgr2=0x7fc3841010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38995e640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc384105ac0 0x7fc3841010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38995e640 1 -- 10.20.193.121:0/2455127011 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc384101600 con 0x7fc384103190 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38995e640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384103190 0x7fc384102a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc37400d8d0 tx=0x7fc37400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc372ffd640 1 -- 10.20.193.121:0/2455127011 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fc374014070 con 0x7fc384103190 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3841018f0 con 0x7fc384103190 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc372ffd640 1 -- 10.20.193.121:0/2455127011 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fc3740044a0 con 0x7fc384103190 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc372ffd640 1 -- 10.20.193.121:0/2455127011 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fc374005020 con 0x7fc384103190 2026-02-13T10:37:59.262 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.258+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc384101f00 con 0x7fc384103190 2026-02-13T10:37:59.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.259+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc384103570 con 0x7fc384103190 2026-02-13T10:37:59.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.259+0000 7fc372ffd640 1 -- 10.20.193.121:0/2455127011 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fc3740051c0 con 0x7fc384103190 2026-02-13T10:37:59.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.260+0000 7fc372ffd640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc360077510 0x7fc3600799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.260+0000 7fc372ffd640 1 -- 10.20.193.121:0/2455127011 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fc3740985e0 con 0x7fc384103190 2026-02-13T10:37:59.264 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.260+0000 7fc38915d640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc360077510 0x7fc3600799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.264 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.265 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.265 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.265 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.265 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.266 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.260+0000 7fc38915d640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc360077510 0x7fc3600799d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc384102780 tx=0x7fc37803a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.266 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.262+0000 7fc372ffd640 1 -- 10.20.193.121:0/2455127011 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc374066770 con 0x7fc384103190 2026-02-13T10:37:59.292 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.290+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f6f541960e0 con 0x7f6f54105b40 2026-02-13T10:37:59.294 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.293+0000 7f6f49ffb640 1 -- 10.20.193.121:0/2611132715 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v29)=0 v29) ==== 125+0+0 (secure 0 0 0) 0x7f6f3c08b380 con 0x7f6f54105b40 2026-02-13T10:37:59.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.295+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6f300774c0 msgr2=0x7f6f30079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.295+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6f300774c0 0x7f6f30079980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f541059a0 tx=0x7f6f44005d90 comp rx=0 tx=0).stop 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.295+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 msgr2=0x7f6f5407b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.295+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f5407b370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c038660 tx=0x7f6f3c002f60 comp rx=0 tx=0).stop 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.296+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 shutdown_connections 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.296+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6f300774c0 0x7f6f30079980 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.296+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6f54105b40 0x7f6f5407b370 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.296+0000 7f6f5ad01640 1 --2- 10.20.193.121:0/2611132715 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6f54104940 0x7f6f5407ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.296+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 >> 10.20.193.121:0/2611132715 conn(0x7f6f541000f0 msgr2=0x7f6f54109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.296+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 shutdown_connections 2026-02-13T10:37:59.298 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.296+0000 7f6f5ad01640 1 -- 10.20.193.121:0/2611132715 wait complete. 2026-02-13T10:37:59.338 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.341 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e9ef2631-08c7-11f1-a2b3-d404e6e7d460', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='trial121') 2026-02-13T10:37:59.352 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.350+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fc3841a23b0 con 0x7fc384103190 2026-02-13T10:37:59.354 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.353+0000 7fc372ffd640 1 -- 10.20.193.121:0/2455127011 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7fc37406b620 con 0x7fc384103190 2026-02-13T10:37:59.357 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc360077510 msgr2=0x7fc3600799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.357 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc360077510 0x7fc3600799d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc384102780 tx=0x7fc37803a040 comp rx=0 tx=0).stop 2026-02-13T10:37:59.357 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384103190 msgr2=0x7fc384102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.357 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384103190 0x7fc384102a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc37400d8d0 tx=0x7fc37400dda0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.357 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 shutdown_connections 2026-02-13T10:37:59.358 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc360077510 0x7fc3600799d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.358 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc384105ac0 0x7fc3841010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.358 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 --2- 10.20.193.121:0/2455127011 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc384103190 0x7fc384102a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.358 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 >> 10.20.193.121:0/2455127011 conn(0x7fc3840fcd30 msgr2=0x7fc3840ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.358 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 shutdown_connections 2026-02-13T10:37:59.358 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.355+0000 7fc38bbe9640 1 -- 10.20.193.121:0/2455127011 wait complete. 2026-02-13T10:37:59.396 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config dump 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: Reconfiguring mgr.trial121.tkoihk (unknown last config time)... 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: Reconfiguring daemon mgr.trial121.tkoihk on trial121 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/330903944' entity='client.admin' 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: Reconfiguring ceph-exporter.trial121 (monmap changed)... 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial121", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: Reconfiguring daemon ceph-exporter.trial121 on trial121 2026-02-13T10:37:59.427 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3389773428' entity='client.admin' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1934310316' entity='client.admin' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2793272989' entity='client.admin' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/560264047' entity='client.admin' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2361331655' entity='client.admin' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3869377632' entity='client.admin' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3671126764' entity='client.admin' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: Reconfiguring crash.trial121 (monmap changed)... 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.trial121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:59.428 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: Reconfiguring daemon crash.trial121 on trial121 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1419093118' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2637472435' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1686198807' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2867415402' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/442023471' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3702850914' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3158643384' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2056664596' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2343501013' entity='client.admin' 2026-02-13T10:37:59.429 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:37:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2956829499' entity='client.admin' 2026-02-13T10:37:59.540 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: Reconfiguring mgr.trial121.tkoihk (unknown last config time)... 2026-02-13T10:37:59.540 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: Reconfiguring daemon mgr.trial121.tkoihk on trial121 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/330903944' entity='client.admin' 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: Reconfiguring ceph-exporter.trial121 (monmap changed)... 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial121", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: Reconfiguring daemon ceph-exporter.trial121 on trial121 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3389773428' entity='client.admin' 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1934310316' entity='client.admin' 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2793272989' entity='client.admin' 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/560264047' entity='client.admin' 2026-02-13T10:37:59.541 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2361331655' entity='client.admin' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3869377632' entity='client.admin' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3671126764' entity='client.admin' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: Reconfiguring crash.trial121 (monmap changed)... 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.trial121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: Reconfiguring daemon crash.trial121 on trial121 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1419093118' entity='client.admin' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2637472435' entity='client.admin' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1686198807' entity='client.admin' 2026-02-13T10:37:59.542 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2867415402' entity='client.admin' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/442023471' entity='client.admin' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3702850914' entity='client.admin' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3158643384' entity='client.admin' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2056664596' entity='client.admin' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2343501013' entity='client.admin' 2026-02-13T10:37:59.543 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:37:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2956829499' entity='client.admin' 2026-02-13T10:37:59.721 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:37:59.821 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 -- 10.20.193.121:0/3807229484 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41026b0 msgr2=0x7fbdd4102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.821 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/3807229484 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41026b0 0x7fbdd4102ab0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00099b0 tx=0x7fbdc002f220 comp rx=0 tx=0).stop 2026-02-13T10:37:59.822 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 -- 10.20.193.121:0/3807229484 shutdown_connections 2026-02-13T10:37:59.822 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/3807229484 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbdd41038b0 0x7fbdd4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.822 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/3807229484 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41026b0 0x7fbdd4102ab0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.822 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 -- 10.20.193.121:0/3807229484 >> 10.20.193.121:0/3807229484 conn(0x7fbdd40fde60 msgr2=0x7fbdd4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.822 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 -- 10.20.193.121:0/3807229484 shutdown_connections 2026-02-13T10:37:59.822 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.817+0000 7fbdd9b18640 1 -- 10.20.193.121:0/3807229484 wait complete. 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 Processor -- start 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 -- start start 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbdd41026b0 0x7fbdd419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41038b0 0x7fbdd419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410b3a0 con 0x7fbdd41038b0 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410b220 con 0x7fbdd41026b0 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd2ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41038b0 0x7fbdd419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd2ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41038b0 0x7fbdd419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57306/0 (socket says 10.20.193.121:57306) 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd2ffd640 1 -- 10.20.193.121:0/1324120296 learned_addr learned my addr 10.20.193.121:0/1324120296 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:37:59.823 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd2ffd640 1 -- 10.20.193.121:0/1324120296 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbdd41026b0 msgr2=0x7fbdd419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd2ffd640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbdd41026b0 0x7fbdd419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd2ffd640 1 -- 10.20.193.121:0/1324120296 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd419ad30 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd2ffd640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41038b0 0x7fbdd419a6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c8f0 tx=0x7fbdc400cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd0ff9640 1 -- 10.20.193.121:0/1324120296 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fbdc4007c30 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd0ff9640 1 -- 10.20.193.121:0/1324120296 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fbdc4007dd0 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd419f770 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd0ff9640 1 -- 10.20.193.121:0/1324120296 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fbdc4018840 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.818+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd419fd80 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.819+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd4102ab0 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.819+0000 7fbdd0ff9640 1 -- 10.20.193.121:0/1324120296 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7fbdc40189e0 con 0x7fbdd41038b0 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.820+0000 7fbdd0ff9640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbda80774c0 0x7fbda8079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:37:59.824 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.820+0000 7fbdd0ff9640 1 -- 10.20.193.121:0/1324120296 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7fbdc4098d70 con 0x7fbdd41038b0 2026-02-13T10:37:59.825 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.820+0000 7fbdd37fe640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbda80774c0 0x7fbda8079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:37:59.825 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.820+0000 7fbdd37fe640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbda80774c0 0x7fbda8079980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00097c0 tx=0x7fbdc0005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:37:59.825 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.821+0000 7fbdd0ff9640 1 -- 10.20.193.121:0/1324120296 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbdc4014030 con 0x7fbdd41038b0 2026-02-13T10:37:59.907 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.906+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbdd4193b60 con 0x7fbdd41038b0 2026-02-13T10:37:59.908 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.906+0000 7fbdd0ff9640 1 -- 10.20.193.121:0/1324120296 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) ==== 59+0+18512 (secure 0 0 0) 0x7fbdc401f300 con 0x7fbdd41038b0 2026-02-13T10:37:59.909 INFO:teuthology.orchestra.run.trial121.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-13T10:37:59.909 INFO:teuthology.orchestra.run.trial121.stdout:global dev auth_debug true 2026-02-13T10:37:59.909 INFO:teuthology.orchestra.run.trial121.stdout:global basic container_image quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4 * 2026-02-13T10:37:59.909 INFO:teuthology.orchestra.run.trial121.stdout:global dev debug_asserts_on_shutdown true 2026-02-13T10:37:59.909 INFO:teuthology.orchestra.run.trial121.stdout:global basic log_to_file true 2026-02-13T10:37:59.909 INFO:teuthology.orchestra.run.trial121.stdout:global basic log_to_journald false 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global basic log_to_stderr false 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_allow_pool_delete true 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_clock_drift_allowed 1.000000 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_cluster_log_to_file true 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_max_pg_per_osd 10000 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_pg_warn_min_per_osd 0 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2026-02-13T10:37:59.910 INFO:teuthology.orchestra.run.trial121.stdout:global advanced mon_warn_on_too_few_osds false 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:global dev ms_die_on_bug true 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:global dev ms_die_on_old_message true 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:global advanced osd_pool_default_erasure_code_profile plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced auth_service_ticket_ttl 240.000000 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced debug_mon 20/20 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced debug_ms 1/1 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced debug_paxos 20/20 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_data_avail_warn 5 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_down_mkfs_grace 300 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_mgr_mkfs_grace 240 2026-02-13T10:37:59.911 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_op_complaint_time 120 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mon dev mon_osd_prime_pg_temp true 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_osd_reporter_subtree_level osd 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mon advanced public_network 10.20.192.0/20 * 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced client_mount_timeout 30 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced debug_client 20/20 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced debug_mgr 20/20 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced debug_ms 1/1 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2026-02-13T10:37:59.912 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mgr/cephadm/container_init True * 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mgr/dashboard/GRAFANA_API_SSL_VERIFY false * 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:mgr dev mon_warn_on_pool_no_app false 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced bdev_async_discard true 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:osd dev bdev_debug_aio true 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced bdev_enable_discard true 2026-02-13T10:37:59.913 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced bluestore_allocator bitmap * 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd dev bluestore_block_size 96636764160 * 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd dev bluestore_fsck_on_mount true 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced debug_bluefs 1/20 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced debug_bluestore 1/20 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced debug_ms 1/1 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced debug_osd 20/20 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced debug_rocksdb 4/10 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_class_default_list * * 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_class_load_list * * 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_debug_misdirected_ops true 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_debug_op_order true 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_debug_pg_log_writeout true 2026-02-13T10:37:59.914 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_debug_shutdown true 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_debug_verify_cached_snaps true 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_debug_verify_missing_on_start true 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_debug_verify_stray_on_activate true 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_mclock_profile high_recovery_ops 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd dev osd_mclock_skip_benchmark true 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_memory_target_autotune true 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_objectstore bluestore * 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_op_complaint_time 180.000000 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_op_queue debug_random * 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_op_queue_cut_off debug_random * 2026-02-13T10:37:59.915 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_recover_clone_overlap true 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_recovery_max_chunk 1048576 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_scrub_load_threshold 5.000000 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_scrub_max_interval 600.000000 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:osd advanced osd_shutdown_pgref_assert true 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds advanced debug_mds 20/20 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds advanced debug_mds_balancer 20/20 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds advanced debug_ms 2/2 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds dev mds_debug_frag true 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds dev mds_debug_scatterstat true 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds advanced mds_op_complaint_time 180.000000 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds dev mds_verify_scatter true 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds advanced osd_op_complaint_time 180.000000 2026-02-13T10:37:59.916 INFO:teuthology.orchestra.run.trial121.stdout:mds advanced rados_mon_op_timeout 900 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:mds advanced rados_osd_op_timeout 900 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client advanced client_mount_timeout 600 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client advanced debug_client 20/20 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client advanced debug_ms 1/1 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client advanced rados_mon_op_timeout 900 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client advanced rados_osd_op_timeout 900 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client.rgw advanced rgw_cache_enabled true 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client.rgw advanced rgw_enable_ops_log true 2026-02-13T10:37:59.917 INFO:teuthology.orchestra.run.trial121.stdout:client.rgw advanced rgw_enable_usage_log true 2026-02-13T10:37:59.918 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbda80774c0 msgr2=0x7fbda8079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.918 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbda80774c0 0x7fbda8079980 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00097c0 tx=0x7fbdc0005e50 comp rx=0 tx=0).stop 2026-02-13T10:37:59.918 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41038b0 msgr2=0x7fbdd419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41038b0 0x7fbdd419a6b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c8f0 tx=0x7fbdc400cdc0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 shutdown_connections 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fbda80774c0 0x7fbda8079980 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fbdd41038b0 0x7fbdd419a6b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 --2- 10.20.193.121:0/1324120296 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fbdd41026b0 0x7fbdd419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 >> 10.20.193.121:0/1324120296 conn(0x7fbdd40fde60 msgr2=0x7fbdd4106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 shutdown_connections 2026-02-13T10:37:59.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:37:59.908+0000 7fbdd9b18640 1 -- 10.20.193.121:0/1324120296 wait complete. 2026-02-13T10:37:59.946 INFO:tasks.cephadm:Deploying OSDs... 2026-02-13T10:37:59.947 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:37:59.947 DEBUG:teuthology.orchestra.run.trial121:> dd if=/scratch_devs of=/dev/stdout 2026-02-13T10:37:59.964 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-13T10:37:59.964 DEBUG:teuthology.orchestra.run.trial121:> stat /dev/vg_nvme/lv_1 2026-02-13T10:38:00.022 INFO:teuthology.orchestra.run.trial121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-13T10:38:00.022 INFO:teuthology.orchestra.run.trial121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:00.023 INFO:teuthology.orchestra.run.trial121.stdout:Device: 6h/6d Inode: 910 Links: 1 2026-02-13T10:38:00.023 INFO:teuthology.orchestra.run.trial121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:00.023 INFO:teuthology.orchestra.run.trial121.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:00.023 INFO:teuthology.orchestra.run.trial121.stdout:Access: 2026-02-13 10:37:41.656540476 +0000 2026-02-13T10:38:00.023 INFO:teuthology.orchestra.run.trial121.stdout:Modify: 2026-02-13 10:35:28.393975738 +0000 2026-02-13T10:38:00.023 INFO:teuthology.orchestra.run.trial121.stdout:Change: 2026-02-13 10:35:28.393975738 +0000 2026-02-13T10:38:00.023 INFO:teuthology.orchestra.run.trial121.stdout: Birth: 2026-02-13 10:35:28.393975738 +0000 2026-02-13T10:38:00.023 DEBUG:teuthology.orchestra.run.trial121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-13T10:38:00.089 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records in 2026-02-13T10:38:00.089 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records out 2026-02-13T10:38:00.090 INFO:teuthology.orchestra.run.trial121.stderr:512 bytes copied, 0.000178661 s, 2.9 MB/s 2026-02-13T10:38:00.091 DEBUG:teuthology.orchestra.run.trial121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-13T10:38:00.151 DEBUG:teuthology.orchestra.run.trial121:> stat /dev/vg_nvme/lv_2 2026-02-13T10:38:00.208 INFO:teuthology.orchestra.run.trial121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-13T10:38:00.208 INFO:teuthology.orchestra.run.trial121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:00.209 INFO:teuthology.orchestra.run.trial121.stdout:Device: 6h/6d Inode: 941 Links: 1 2026-02-13T10:38:00.209 INFO:teuthology.orchestra.run.trial121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:00.209 INFO:teuthology.orchestra.run.trial121.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:00.209 INFO:teuthology.orchestra.run.trial121.stdout:Access: 2026-02-13 10:37:41.656540476 +0000 2026-02-13T10:38:00.209 INFO:teuthology.orchestra.run.trial121.stdout:Modify: 2026-02-13 10:35:28.620971879 +0000 2026-02-13T10:38:00.209 INFO:teuthology.orchestra.run.trial121.stdout:Change: 2026-02-13 10:35:28.620971879 +0000 2026-02-13T10:38:00.209 INFO:teuthology.orchestra.run.trial121.stdout: Birth: 2026-02-13 10:35:28.620971879 +0000 2026-02-13T10:38:00.209 DEBUG:teuthology.orchestra.run.trial121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-13T10:38:00.272 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records in 2026-02-13T10:38:00.272 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records out 2026-02-13T10:38:00.272 INFO:teuthology.orchestra.run.trial121.stderr:512 bytes copied, 0.000151752 s, 3.4 MB/s 2026-02-13T10:38:00.273 DEBUG:teuthology.orchestra.run.trial121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-13T10:38:00.332 DEBUG:teuthology.orchestra.run.trial121:> stat /dev/vg_nvme/lv_3 2026-02-13T10:38:00.389 INFO:teuthology.orchestra.run.trial121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout:Device: 6h/6d Inode: 967 Links: 1 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout:Access: 2026-02-13 10:37:41.656540476 +0000 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout:Modify: 2026-02-13 10:35:28.883967406 +0000 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout:Change: 2026-02-13 10:35:28.883967406 +0000 2026-02-13T10:38:00.390 INFO:teuthology.orchestra.run.trial121.stdout: Birth: 2026-02-13 10:35:28.883967406 +0000 2026-02-13T10:38:00.391 DEBUG:teuthology.orchestra.run.trial121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-13T10:38:00.455 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records in 2026-02-13T10:38:00.455 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records out 2026-02-13T10:38:00.455 INFO:teuthology.orchestra.run.trial121.stderr:512 bytes copied, 0.000196434 s, 2.6 MB/s 2026-02-13T10:38:00.456 DEBUG:teuthology.orchestra.run.trial121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-13T10:38:00.512 DEBUG:teuthology.orchestra.run.trial121:> stat /dev/vg_nvme/lv_4 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: Reconfiguring alertmanager.trial121 (dependencies changed)... 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: Reconfiguring daemon alertmanager.trial121 on trial121 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2611132715' entity='client.admin' 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2455127011' entity='client.admin' 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: Reconfiguring grafana.trial121 (dependencies changed)... 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: Reconfiguring daemon grafana.trial121 on trial121 2026-02-13T10:38:00.532 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:00 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1324120296' entity='client.admin' cmd=[{"prefix": "config dump"}]: dispatch 2026-02-13T10:38:00.547 INFO:teuthology.orchestra.run.trial121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-13T10:38:00.547 INFO:teuthology.orchestra.run.trial121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:00.547 INFO:teuthology.orchestra.run.trial121.stdout:Device: 6h/6d Inode: 994 Links: 1 2026-02-13T10:38:00.547 INFO:teuthology.orchestra.run.trial121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:00.547 INFO:teuthology.orchestra.run.trial121.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:00.548 INFO:teuthology.orchestra.run.trial121.stdout:Access: 2026-02-13 10:37:41.656540476 +0000 2026-02-13T10:38:00.548 INFO:teuthology.orchestra.run.trial121.stdout:Modify: 2026-02-13 10:35:29.129963224 +0000 2026-02-13T10:38:00.548 INFO:teuthology.orchestra.run.trial121.stdout:Change: 2026-02-13 10:35:29.129963224 +0000 2026-02-13T10:38:00.548 INFO:teuthology.orchestra.run.trial121.stdout: Birth: 2026-02-13 10:35:29.129963224 +0000 2026-02-13T10:38:00.548 DEBUG:teuthology.orchestra.run.trial121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-13T10:38:00.612 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records in 2026-02-13T10:38:00.612 INFO:teuthology.orchestra.run.trial121.stderr:1+0 records out 2026-02-13T10:38:00.612 INFO:teuthology.orchestra.run.trial121.stderr:512 bytes copied, 0.000157312 s, 3.3 MB/s 2026-02-13T10:38:00.613 DEBUG:teuthology.orchestra.run.trial121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-13T10:38:00.670 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:38:00.670 DEBUG:teuthology.orchestra.run.trial163:> dd if=/scratch_devs of=/dev/stdout 2026-02-13T10:38:00.689 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-13T10:38:00.689 DEBUG:teuthology.orchestra.run.trial163:> stat /dev/vg_nvme/lv_1 2026-02-13T10:38:00.746 INFO:teuthology.orchestra.run.trial163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-13T10:38:00.746 INFO:teuthology.orchestra.run.trial163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:00.746 INFO:teuthology.orchestra.run.trial163.stdout:Device: 6h/6d Inode: 910 Links: 1 2026-02-13T10:38:00.746 INFO:teuthology.orchestra.run.trial163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:00.747 INFO:teuthology.orchestra.run.trial163.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:00.747 INFO:teuthology.orchestra.run.trial163.stdout:Access: 2026-02-13 10:37:51.745193819 +0000 2026-02-13T10:38:00.747 INFO:teuthology.orchestra.run.trial163.stdout:Modify: 2026-02-13 10:35:27.580024976 +0000 2026-02-13T10:38:00.747 INFO:teuthology.orchestra.run.trial163.stdout:Change: 2026-02-13 10:35:27.580024976 +0000 2026-02-13T10:38:00.747 INFO:teuthology.orchestra.run.trial163.stdout: Birth: 2026-02-13 10:35:27.580024976 +0000 2026-02-13T10:38:00.747 DEBUG:teuthology.orchestra.run.trial163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-13T10:38:00.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: Reconfiguring alertmanager.trial121 (dependencies changed)... 2026-02-13T10:38:00.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: Reconfiguring daemon alertmanager.trial121 on trial121 2026-02-13T10:38:00.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2611132715' entity='client.admin' 2026-02-13T10:38:00.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2455127011' entity='client.admin' 2026-02-13T10:38:00.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:00.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:00.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: Reconfiguring grafana.trial121 (dependencies changed)... 2026-02-13T10:38:00.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: Reconfiguring daemon grafana.trial121 on trial121 2026-02-13T10:38:00.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:00 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1324120296' entity='client.admin' cmd=[{"prefix": "config dump"}]: dispatch 2026-02-13T10:38:00.810 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records in 2026-02-13T10:38:00.810 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records out 2026-02-13T10:38:00.810 INFO:teuthology.orchestra.run.trial163.stderr:512 bytes copied, 0.000152403 s, 3.4 MB/s 2026-02-13T10:38:00.811 DEBUG:teuthology.orchestra.run.trial163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-13T10:38:00.868 DEBUG:teuthology.orchestra.run.trial163:> stat /dev/vg_nvme/lv_2 2026-02-13T10:38:00.924 INFO:teuthology.orchestra.run.trial163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout:Device: 6h/6d Inode: 938 Links: 1 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout:Access: 2026-02-13 10:37:51.745193819 +0000 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout:Modify: 2026-02-13 10:35:27.810019502 +0000 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout:Change: 2026-02-13 10:35:27.810019502 +0000 2026-02-13T10:38:00.925 INFO:teuthology.orchestra.run.trial163.stdout: Birth: 2026-02-13 10:35:27.810019502 +0000 2026-02-13T10:38:00.926 DEBUG:teuthology.orchestra.run.trial163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-13T10:38:00.987 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records in 2026-02-13T10:38:00.987 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records out 2026-02-13T10:38:00.988 INFO:teuthology.orchestra.run.trial163.stderr:512 bytes copied, 0.00012875 s, 4.0 MB/s 2026-02-13T10:38:00.989 DEBUG:teuthology.orchestra.run.trial163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-13T10:38:01.046 DEBUG:teuthology.orchestra.run.trial163:> stat /dev/vg_nvme/lv_3 2026-02-13T10:38:01.102 INFO:teuthology.orchestra.run.trial163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout:Device: 6h/6d Inode: 966 Links: 1 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout:Access: 2026-02-13 10:37:51.746193800 +0000 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout:Modify: 2026-02-13 10:35:28.054013694 +0000 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout:Change: 2026-02-13 10:35:28.054013694 +0000 2026-02-13T10:38:01.103 INFO:teuthology.orchestra.run.trial163.stdout: Birth: 2026-02-13 10:35:28.054013694 +0000 2026-02-13T10:38:01.104 DEBUG:teuthology.orchestra.run.trial163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-13T10:38:01.166 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records in 2026-02-13T10:38:01.167 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records out 2026-02-13T10:38:01.167 INFO:teuthology.orchestra.run.trial163.stderr:512 bytes copied, 0.00013941 s, 3.7 MB/s 2026-02-13T10:38:01.168 DEBUG:teuthology.orchestra.run.trial163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-13T10:38:01.227 DEBUG:teuthology.orchestra.run.trial163:> stat /dev/vg_nvme/lv_4 2026-02-13T10:38:01.284 INFO:teuthology.orchestra.run.trial163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-13T10:38:01.284 INFO:teuthology.orchestra.run.trial163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T10:38:01.284 INFO:teuthology.orchestra.run.trial163.stdout:Device: 6h/6d Inode: 997 Links: 1 2026-02-13T10:38:01.284 INFO:teuthology.orchestra.run.trial163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T10:38:01.285 INFO:teuthology.orchestra.run.trial163.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T10:38:01.285 INFO:teuthology.orchestra.run.trial163.stdout:Access: 2026-02-13 10:37:51.746193800 +0000 2026-02-13T10:38:01.285 INFO:teuthology.orchestra.run.trial163.stdout:Modify: 2026-02-13 10:35:28.315007483 +0000 2026-02-13T10:38:01.285 INFO:teuthology.orchestra.run.trial163.stdout:Change: 2026-02-13 10:35:28.315007483 +0000 2026-02-13T10:38:01.285 INFO:teuthology.orchestra.run.trial163.stdout: Birth: 2026-02-13 10:35:28.315007483 +0000 2026-02-13T10:38:01.285 DEBUG:teuthology.orchestra.run.trial163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-13T10:38:01.348 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records in 2026-02-13T10:38:01.348 INFO:teuthology.orchestra.run.trial163.stderr:1+0 records out 2026-02-13T10:38:01.348 INFO:teuthology.orchestra.run.trial163.stderr:512 bytes copied, 0.000173312 s, 3.0 MB/s 2026-02-13T10:38:01.349 DEBUG:teuthology.orchestra.run.trial163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-13T10:38:01.407 INFO:tasks.cephadm:Deploying osd.0 on trial121 with /dev/vg_nvme/lv_4... 2026-02-13T10:38:01.408 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-13T10:38:01.680 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.680 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.680 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: Reconfiguring prometheus.trial121 (dependencies changed)... 2026-02-13T10:38:01.680 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:01.680 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: Reconfiguring daemon prometheus.trial121 on trial121 2026-02-13T10:38:01.681 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.681 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.681 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2026-02-13T10:38:01.681 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:01 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:01.696 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: Reconfiguring prometheus.trial121 (dependencies changed)... 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: Reconfiguring daemon prometheus.trial121 on trial121 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2026-02-13T10:38:01.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:01 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:02.454 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:38:02.471 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial121:vg_nvme/lv_4 --skip-validation 2026-02-13T10:38:02.581 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: Reconfiguring ceph-exporter.trial163 (monmap changed)... 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: Reconfiguring daemon ceph-exporter.trial163 on trial163 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: Reconfiguring crash.trial163 (monmap changed)... 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.trial163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: Reconfiguring daemon crash.trial163 on trial163 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: Reconfiguring mgr.trial163.xfyaeo (monmap changed)... 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.trial163.xfyaeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr services"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: Reconfiguring daemon mgr.trial163.xfyaeo on trial163 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial121:9093"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://trial121:3000"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://trial121:9095"}]: dispatch 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.582 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:02 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:02.772 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: Reconfiguring ceph-exporter.trial163 (monmap changed)... 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: Reconfiguring daemon ceph-exporter.trial163 on trial163 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: Reconfiguring crash.trial163 (monmap changed)... 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.trial163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: Reconfiguring daemon crash.trial163 on trial163 2026-02-13T10:38:02.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: Reconfiguring mgr.trial163.xfyaeo (monmap changed)... 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.trial163.xfyaeo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mgr services"}]: dispatch 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: Reconfiguring daemon mgr.trial163.xfyaeo on trial163 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial121:9093"}]: dispatch 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://trial121:3000"}]: dispatch 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://trial121:9095"}]: dispatch 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:02.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:02 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:02.894 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 -- 10.20.193.121:0/3065086011 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4e50104950 msgr2=0x7f4e50104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 --2- 10.20.193.121:0/3065086011 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4e50104950 0x7f4e50104d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4e400099b0 tx=0x7f4e4002f220 comp rx=0 tx=0).stop 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 -- 10.20.193.121:0/3065086011 shutdown_connections 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 --2- 10.20.193.121:0/3065086011 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4e50105b50 0x7f4e50105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 --2- 10.20.193.121:0/3065086011 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4e50104950 0x7f4e50104d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 -- 10.20.193.121:0/3065086011 >> 10.20.193.121:0/3065086011 conn(0x7f4e501000e0 msgr2=0x7f4e50102520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 -- 10.20.193.121:0/3065086011 shutdown_connections 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 -- 10.20.193.121:0/3065086011 wait complete. 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 Processor -- start 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.891+0000 7f4e57805640 1 -- start start 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e57805640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4e50104950 0x7f4e5019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:02.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e57805640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4e50105b50 0x7f4e5019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e57805640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5010d620 con 0x7f4e50104950 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e57805640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5010d4a0 con 0x7f4e50105b50 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e5557a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4e50104950 0x7f4e5019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e5557a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4e50104950 0x7f4e5019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57330/0 (socket says 10.20.193.121:57330) 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e5557a640 1 -- 10.20.193.121:0/2928120217 learned_addr learned my addr 10.20.193.121:0/2928120217 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e54d79640 1 --2- 10.20.193.121:0/2928120217 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4e50105b50 0x7f4e5019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e5557a640 1 -- 10.20.193.121:0/2928120217 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4e50105b50 msgr2=0x7f4e5019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e5557a640 1 --2- 10.20.193.121:0/2928120217 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4e50105b50 0x7f4e5019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e5557a640 1 -- 10.20.193.121:0/2928120217 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5019cfa0 con 0x7f4e50104950 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e5557a640 1 --2- 10.20.193.121:0/2928120217 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4e50104950 0x7f4e5019c3e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e4002f730 tx=0x7f4e40038660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:02.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e3e7fc640 1 -- 10.20.193.121:0/2928120217 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4e40046070 con 0x7f4e50104950 2026-02-13T10:38:02.897 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e57805640 1 -- 10.20.193.121:0/2928120217 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e501a1980 con 0x7f4e50104950 2026-02-13T10:38:02.897 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e3e7fc640 1 -- 10.20.193.121:0/2928120217 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f4e400044b0 con 0x7f4e50104950 2026-02-13T10:38:02.897 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e3e7fc640 1 -- 10.20.193.121:0/2928120217 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4e40041a50 con 0x7f4e50104950 2026-02-13T10:38:02.897 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.892+0000 7f4e57805640 1 -- 10.20.193.121:0/2928120217 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501a1f90 con 0x7f4e50104950 2026-02-13T10:38:02.897 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.893+0000 7f4e57805640 1 -- 10.20.193.121:0/2928120217 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e50104d50 con 0x7f4e50104950 2026-02-13T10:38:02.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.893+0000 7f4e3e7fc640 1 -- 10.20.193.121:0/2928120217 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f4e40038770 con 0x7f4e50104950 2026-02-13T10:38:02.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.894+0000 7f4e3e7fc640 1 --2- 10.20.193.121:0/2928120217 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4e2c077510 0x7f4e2c0799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:02.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.894+0000 7f4e54d79640 1 --2- 10.20.193.121:0/2928120217 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4e2c077510 0x7f4e2c0799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:02.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.894+0000 7f4e3e7fc640 1 -- 10.20.193.121:0/2928120217 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f4e4003e070 con 0x7f4e50104950 2026-02-13T10:38:02.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.894+0000 7f4e54d79640 1 --2- 10.20.193.121:0/2928120217 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4e2c077510 0x7f4e2c0799d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e5019d890 tx=0x7f4e4400a5c0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:02.898 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:02.896+0000 7f4e3e7fc640 1 -- 10.20.193.121:0/2928120217 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4e40050020 con 0x7f4e50104950 2026-02-13T10:38:02.988 INFO:teuthology.orchestra.run.trial121.stderr:--skip-validation not valid: --skip-validation not in raw|lvm 2026-02-13T10:38:02.988 INFO:teuthology.orchestra.run.trial121.stderr:Invalid command: unused arguments: ['--skip-validation'] 2026-02-13T10:38:02.988 INFO:teuthology.orchestra.run.trial121.stderr:orch daemon add osd [] [] : Create OSD daemon(s) on specified host and device(s) (e.g., ceph orch daemon add osd myhost:/dev/sdb) 2026-02-13T10:38:02.988 INFO:teuthology.orchestra.run.trial121.stderr:Error EINVAL: invalid command 2026-02-13T10:38:03.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-13T10:38:03.024 WARNING:tasks.cephadm:--skip-validation falied with error Command failed on trial121 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial121:vg_nvme/lv_4 --skip-validation'. Retrying without it 2026-02-13T10:38:03.024 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial121:vg_nvme/lv_4 2026-02-13T10:38:03.313 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:38:03.421 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 -- 10.20.193.121:0/2944891040 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 msgr2=0x7f7c28077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:03.421 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 --2- 10.20.193.121:0/2944891040 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c28077ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7c180099b0 tx=0x7f7c1802f220 comp rx=0 tx=0).stop 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 -- 10.20.193.121:0/2944891040 shutdown_connections 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 --2- 10.20.193.121:0/2944891040 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7c28078cf0 0x7f7c28079150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 --2- 10.20.193.121:0/2944891040 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c28077ea0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 -- 10.20.193.121:0/2944891040 >> 10.20.193.121:0/2944891040 conn(0x7f7c281000f0 msgr2=0x7f7c28102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 -- 10.20.193.121:0/2944891040 shutdown_connections 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 -- 10.20.193.121:0/2944891040 wait complete. 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 Processor -- start 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.418+0000 7f7c2f621640 1 -- start start 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2f621640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c281a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:03.422 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2f621640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7c28078cf0 0x7f7c281a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2f621640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28111b00 con 0x7f7c28077aa0 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2f621640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28111980 con 0x7f7c28078cf0 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2d396640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c281a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2d396640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c281a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57346/0 (socket says 10.20.193.121:57346) 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2d396640 1 -- 10.20.193.121:0/3810466322 learned_addr learned my addr 10.20.193.121:0/3810466322 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2d396640 1 -- 10.20.193.121:0/3810466322 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7c28078cf0 msgr2=0x7f7c281a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2d396640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7c28078cf0 0x7f7c281a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2d396640 1 -- 10.20.193.121:0/3810466322 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c281a13f0 con 0x7f7c28077aa0 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2d396640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c281a08c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c18038660 tx=0x7f7c18002f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c167fc640 1 -- 10.20.193.121:0/3810466322 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7c18046070 con 0x7f7c28077aa0 2026-02-13T10:38:03.423 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c167fc640 1 -- 10.20.193.121:0/3810466322 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f7c180044b0 con 0x7f7c28077aa0 2026-02-13T10:38:03.424 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c281a5e00 con 0x7f7c28077aa0 2026-02-13T10:38:03.424 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c167fc640 1 -- 10.20.193.121:0/3810466322 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7c18041a80 con 0x7f7c28077aa0 2026-02-13T10:38:03.424 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c281a6380 con 0x7f7c28077aa0 2026-02-13T10:38:03.424 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.419+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c28077ea0 con 0x7f7c28077aa0 2026-02-13T10:38:03.425 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.420+0000 7f7c167fc640 1 -- 10.20.193.121:0/3810466322 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f7c18038770 con 0x7f7c28077aa0 2026-02-13T10:38:03.425 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.421+0000 7f7c167fc640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c040774c0 0x7f7c04079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:03.425 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.421+0000 7f7c167fc640 1 -- 10.20.193.121:0/3810466322 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1365+0+0 (secure 0 0 0) 0x7f7c1803e070 con 0x7f7c28077aa0 2026-02-13T10:38:03.425 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.421+0000 7f7c2cb95640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c040774c0 0x7f7c04079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:03.425 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.421+0000 7f7c2cb95640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c040774c0 0x7f7c04079980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c281a1d70 tx=0x7f7c1c005d70 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:03.425 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.422+0000 7f7c167fc640 1 -- 10.20.193.121:0/3810466322 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7c18050020 con 0x7f7c28077aa0 2026-02-13T10:38:03.511 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:03.509+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7c280630c0 con 0x7f7c040774c0 2026-02-13T10:38:03.891 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: Reconfiguring mon.trial163 (monmap changed)... 2026-02-13T10:38:03.891 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: Reconfiguring daemon mon.trial163 on trial163 2026-02-13T10:38:03.891 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2026-02-13T10:38:03.891 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial121:9093"}]: dispatch 2026-02-13T10:38:03.891 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2026-02-13T10:38:03.891 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://trial121:3000"}]: dispatch 2026-02-13T10:38:03.891 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://trial121:9095"}]: dispatch 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:03.892 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: Reconfiguring mon.trial163 (monmap changed)... 2026-02-13T10:38:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: Reconfiguring daemon mon.trial163 on trial163 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial121:9093"}]: dispatch 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://trial121:3000"}]: dispatch 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://trial121:9095"}]: dispatch 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:04.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:04.042 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:04.042 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:04.042 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:04.042 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:04.042 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:05.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:04 trial163 ceph-mon[24184]: from='client.14328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:05.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:04 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2099843328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15b6509f-b28f-4bda-bbdc-c0c0b3074fca"}]: dispatch 2026-02-13T10:38:05.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:04 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2099843328' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15b6509f-b28f-4bda-bbdc-c0c0b3074fca"}]': finished 2026-02-13T10:38:05.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:04 trial163 ceph-mon[24184]: osdmap e6: 1 total, 0 up, 1 in 2026-02-13T10:38:05.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:04 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:05.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:04 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/407544371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:04 trial121 ceph-mon[19159]: from='client.14328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:04 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2099843328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15b6509f-b28f-4bda-bbdc-c0c0b3074fca"}]: dispatch 2026-02-13T10:38:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:04 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2099843328' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15b6509f-b28f-4bda-bbdc-c0c0b3074fca"}]': finished 2026-02-13T10:38:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:04 trial121 ceph-mon[19159]: osdmap e6: 1 total, 0 up, 1 in 2026-02-13T10:38:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:04 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:04 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/407544371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:06.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:06 trial121 ceph-mon[19159]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:07.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:06 trial163 ceph-mon[24184]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:08.912 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:08 trial121 ceph-mon[19159]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:09.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:08 trial163 ceph-mon[24184]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:10.816 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:10 trial121 ceph-mon[19159]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:10.816 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:10 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2026-02-13T10:38:10.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:10 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:10.817 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:10 trial121 ceph-mon[19159]: Deploying daemon osd.0 on trial121 2026-02-13T10:38:11.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:10 trial163 ceph-mon[24184]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:11.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:10 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2026-02-13T10:38:11.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:10 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:11.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:10 trial163 ceph-mon[24184]: Deploying daemon osd.0 on trial121 2026-02-13T10:38:12.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:12 trial121 ceph-mon[19159]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:12.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:12 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:12.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:12 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:12.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:12 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:12.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:12 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:12.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:12 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:12.892 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.891+0000 7f7c167fc640 1 -- 10.20.193.121:0/3810466322 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f7c280630c0 con 0x7f7c040774c0 2026-02-13T10:38:12.893 INFO:teuthology.orchestra.run.trial121.stdout:Created osd(s) 0 on host 'trial121' 2026-02-13T10:38:12.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c040774c0 msgr2=0x7f7c04079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:12.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c040774c0 0x7f7c04079980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c281a1d70 tx=0x7f7c1c005d70 comp rx=0 tx=0).stop 2026-02-13T10:38:12.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 msgr2=0x7f7c281a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:12.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c281a08c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c18038660 tx=0x7f7c18002f60 comp rx=0 tx=0).stop 2026-02-13T10:38:12.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 shutdown_connections 2026-02-13T10:38:12.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c040774c0 0x7f7c04079980 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:12.895 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7c28078cf0 0x7f7c281a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:12.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 --2- 10.20.193.121:0/3810466322 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7c28077aa0 0x7f7c281a08c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:12.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 >> 10.20.193.121:0/3810466322 conn(0x7f7c281000f0 msgr2=0x7f7c2810f170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:12.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 shutdown_connections 2026-02-13T10:38:12.896 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:12.893+0000 7f7c2f621640 1 -- 10.20.193.121:0/3810466322 wait complete. 2026-02-13T10:38:12.954 DEBUG:teuthology.orchestra.run.trial121:osd.0> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.0.service 2026-02-13T10:38:12.957 INFO:tasks.cephadm:Deploying osd.1 on trial121 with /dev/vg_nvme/lv_3... 2026-02-13T10:38:12.957 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-13T10:38:13.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:12 trial163 ceph-mon[24184]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:13.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:12 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:13.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:12 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:12 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:12 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:12 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.297 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:38:13.958 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:13 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0[40833]: 2026-02-13T10:38:13.773+0000 7f7007324740 -1 Falling back to public interface 2026-02-13T10:38:13.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:13.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:13 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:13 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:14.110 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:38:14.127 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial121:vg_nvme/lv_3 2026-02-13T10:38:14.415 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- 10.20.193.121:0/2156146780 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4102790 msgr2=0x7f46d4102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 --2- 10.20.193.121:0/2156146780 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4102790 0x7f46d4102c10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f46c80099b0 tx=0x7f46c802f220 comp rx=0 tx=0).stop 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- 10.20.193.121:0/2156146780 shutdown_connections 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 --2- 10.20.193.121:0/2156146780 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4102790 0x7f46d4102c10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 --2- 10.20.193.121:0/2156146780 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f46d4101590 0x7f46d4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- 10.20.193.121:0/2156146780 >> 10.20.193.121:0/2156146780 conn(0x7f46d40fcd20 msgr2=0x7f46d40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- 10.20.193.121:0/2156146780 shutdown_connections 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- 10.20.193.121:0/2156146780 wait complete. 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 Processor -- start 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- start start 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4101590 0x7f46d419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f46d4102790 0x7f46d419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410a280 con 0x7f46d4101590 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46dbd86640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410a100 con 0x7f46d4102790 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46d9afb640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4101590 0x7f46d419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46d9afb640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4101590 0x7f46d419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56554/0 (socket says 10.20.193.121:56554) 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46d9afb640 1 -- 10.20.193.121:0/241476312 learned_addr learned my addr 10.20.193.121:0/241476312 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46d92fa640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f46d4102790 0x7f46d419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46d9afb640 1 -- 10.20.193.121:0/241476312 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f46d4102790 msgr2=0x7f46d419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46d9afb640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f46d4102790 0x7f46d419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.534+0000 7f46d9afb640 1 -- 10.20.193.121:0/241476312 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d419cfe0 con 0x7f46d4101590 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.535+0000 7f46d9afb640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4101590 0x7f46d419c420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e940 tx=0x7f46c400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.535+0000 7f46c2ffd640 1 -- 10.20.193.121:0/241476312 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f46c400cb80 con 0x7f46d4101590 2026-02-13T10:38:14.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.535+0000 7f46c2ffd640 1 -- 10.20.193.121:0/241476312 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f46c4004560 con 0x7f46d4101590 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.535+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d41a1a20 con 0x7f46d4101590 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.535+0000 7f46c2ffd640 1 -- 10.20.193.121:0/241476312 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f46c4010430 con 0x7f46d4101590 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.535+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41a2030 con 0x7f46d4101590 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.535+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4101990 con 0x7f46d4101590 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.536+0000 7f46c2ffd640 1 -- 10.20.193.121:0/241476312 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f46c4020030 con 0x7f46d4101590 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.536+0000 7f46c2ffd640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f46b0077510 0x7f46b00799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.536+0000 7f46c2ffd640 1 -- 10.20.193.121:0/241476312 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1480+0+0 (secure 0 0 0) 0x7f46c4098850 con 0x7f46d4101590 2026-02-13T10:38:14.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.536+0000 7f46d92fa640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f46b0077510 0x7f46b00799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:14.538 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.537+0000 7f46d92fa640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f46b0077510 0x7f46b00799d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f46d419d8d0 tx=0x7f46c803a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:14.539 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.538+0000 7f46c2ffd640 1 -- 10.20.193.121:0/241476312 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f46c4066b70 con 0x7f46d4101590 2026-02-13T10:38:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:14.629+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f46d41a2320 con 0x7f46b0077510 2026-02-13T10:38:15.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:15.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:15.171 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:15.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: from='client.14344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: Detected new or changed devices on trial121 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: Adjusting osd_memory_target on trial121 to 78799M 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1108529388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3e08c6d-c2cb-4d19-988c-7ec1d5d70aa0"}]: dispatch 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1108529388' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3e08c6d-c2cb-4d19-988c-7ec1d5d70aa0"}]': finished 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: osdmap e7: 2 total, 0 up, 2 in 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:16.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:15 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/686920931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: from='client.14344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: Detected new or changed devices on trial121 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: Adjusting osd_memory_target on trial121 to 78799M 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1108529388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3e08c6d-c2cb-4d19-988c-7ec1d5d70aa0"}]: dispatch 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1108529388' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3e08c6d-c2cb-4d19-988c-7ec1d5d70aa0"}]': finished 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: osdmap e7: 2 total, 0 up, 2 in 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:15 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/686920931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:17.207 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:16 trial121 ceph-mon[19159]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:17.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:16 trial163 ceph-mon[24184]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:18.457 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:18 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0[40833]: 2026-02-13T10:38:18.162+0000 7f7007324740 -1 osd.0 0 log_to_monitors true 2026-02-13T10:38:19.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:18 trial163 ceph-mon[24184]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:19.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:18 trial163 ceph-mon[24184]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2026-02-13T10:38:19.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:18 trial121 ceph-mon[19159]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:19.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:18 trial121 ceph-mon[19159]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2026-02-13T10:38:20.125 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:19 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0[40833]: 2026-02-13T10:38:19.852+0000 7f7002a82640 -1 osd.0 0 waiting for initial osdmap 2026-02-13T10:38:20.125 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:19 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0[40833]: 2026-02-13T10:38:19.855+0000 7f6ffb0b5640 -1 osd.0 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T10:38:20.125 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:19 trial121 ceph-mon[19159]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-13T10:38:20.126 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:19 trial121 ceph-mon[19159]: osdmap e8: 2 total, 0 up, 2 in 2026-02-13T10:38:20.126 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:19 trial121 ceph-mon[19159]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial121", "root=default"]}]: dispatch 2026-02-13T10:38:20.126 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:19 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:20.126 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:19 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:20.126 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:19 trial121 ceph-mon[19159]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial121", "root=default"]}]': finished 2026-02-13T10:38:20.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:19 trial163 ceph-mon[24184]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-13T10:38:20.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:19 trial163 ceph-mon[24184]: osdmap e8: 2 total, 0 up, 2 in 2026-02-13T10:38:20.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:19 trial163 ceph-mon[24184]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial121", "root=default"]}]: dispatch 2026-02-13T10:38:20.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:19 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:20.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:19 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:20.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:19 trial163 ceph-mon[24184]: from='osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial121", "root=default"]}]': finished 2026-02-13T10:38:20.939 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:20 trial121 ceph-mon[19159]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:20.939 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:20 trial121 ceph-mon[19159]: osdmap e9: 2 total, 0 up, 2 in 2026-02-13T10:38:20.939 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:20 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:20.939 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:20 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:20.939 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:20 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:21.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:20 trial163 ceph-mon[24184]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:21.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:20 trial163 ceph-mon[24184]: osdmap e9: 2 total, 0 up, 2 in 2026-02-13T10:38:21.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:20 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:21.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:20 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:21.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:20 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: purged_snaps scrub starts 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: purged_snaps scrub ok 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] boot 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: osdmap e10: 2 total, 1 up, 2 in 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2026-02-13T10:38:21.989 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:21 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:22.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: purged_snaps scrub starts 2026-02-13T10:38:22.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: purged_snaps scrub ok 2026-02-13T10:38:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: osd.0 [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] boot 2026-02-13T10:38:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: osdmap e10: 2 total, 1 up, 2 in 2026-02-13T10:38:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2026-02-13T10:38:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2026-02-13T10:38:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:21 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:23.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:22 trial121 ceph-mon[19159]: Deploying daemon osd.1 on trial121 2026-02-13T10:38:23.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:22 trial121 ceph-mon[19159]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:23.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:22 trial163 ceph-mon[24184]: Deploying daemon osd.1 on trial121 2026-02-13T10:38:23.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:22 trial163 ceph-mon[24184]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T10:38:24.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:23 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:24.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:23 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:23 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:23 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:23 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.053 INFO:teuthology.orchestra.run.trial121.stdout:Created osd(s) 1 on host 'trial121' 2026-02-13T10:38:24.053 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.051+0000 7f46c2ffd640 1 -- 10.20.193.121:0/241476312 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f46d41a2320 con 0x7f46b0077510 2026-02-13T10:38:24.055 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f46b0077510 msgr2=0x7f46b00799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:24.055 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f46b0077510 0x7f46b00799d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f46d419d8d0 tx=0x7f46c803a040 comp rx=0 tx=0).stop 2026-02-13T10:38:24.055 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4101590 msgr2=0x7f46d419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:24.055 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4101590 0x7f46d419c420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e940 tx=0x7f46c400ee10 comp rx=0 tx=0).stop 2026-02-13T10:38:24.055 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 shutdown_connections 2026-02-13T10:38:24.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f46b0077510 0x7f46b00799d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:24.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f46d4102790 0x7f46d419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:24.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 --2- 10.20.193.121:0/241476312 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f46d4101590 0x7f46d419c420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:24.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 >> 10.20.193.121:0/241476312 conn(0x7f46d40fcd20 msgr2=0x7f46d406ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:24.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 shutdown_connections 2026-02-13T10:38:24.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:24.053+0000 7f46dbd86640 1 -- 10.20.193.121:0/241476312 wait complete. 2026-02-13T10:38:24.070 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:23 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:24.070 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:23 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.070 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:23 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.070 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:23 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.071 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:23 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:24.092 DEBUG:teuthology.orchestra.run.trial121:osd.1> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.1.service 2026-02-13T10:38:24.095 INFO:tasks.cephadm:Deploying osd.2 on trial121 with /dev/vg_nvme/lv_2... 2026-02-13T10:38:24.095 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-13T10:38:24.425 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:38:25.168 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: pgmap v17: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:25.168 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.168 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:25.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:25.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.169 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:24 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:25.169 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:24 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1[47880]: 2026-02-13T10:38:24.965+0000 7f59fa7f4740 -1 Falling back to public interface 2026-02-13T10:38:25.250 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:38:25.268 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial121:vg_nvme/lv_2 2026-02-13T10:38:25.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: pgmap v17: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:25.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:24 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:25.561 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:38:25.670 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 -- 10.20.193.121:0/808588532 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 msgr2=0x7f2038104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:25.670 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 --2- 10.20.193.121:0/808588532 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f2038104b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f202c0099b0 tx=0x7f202c02f220 comp rx=0 tx=0).stop 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 -- 10.20.193.121:0/808588532 shutdown_connections 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 --2- 10.20.193.121:0/808588532 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2038105950 0x7f2038105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 --2- 10.20.193.121:0/808588532 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f2038104b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 -- 10.20.193.121:0/808588532 >> 10.20.193.121:0/808588532 conn(0x7f20380fff00 msgr2=0x7f2038102320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 -- 10.20.193.121:0/808588532 shutdown_connections 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 -- 10.20.193.121:0/808588532 wait complete. 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 Processor -- start 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.666+0000 7f2037fff640 1 -- start start 2026-02-13T10:38:25.671 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2037fff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f203819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2037fff640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2038105950 0x7f203819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2037fff640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f203810d420 con 0x7f2038104750 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2037fff640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f203810d2a0 con 0x7f2038105950 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2036ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f203819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2036ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f203819c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:39280/0 (socket says 10.20.193.121:39280) 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2036ffd640 1 -- 10.20.193.121:0/906667053 learned_addr learned my addr 10.20.193.121:0/906667053 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2036ffd640 1 -- 10.20.193.121:0/906667053 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2038105950 msgr2=0x7f203819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2036ffd640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2038105950 0x7f203819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2036ffd640 1 -- 10.20.193.121:0/906667053 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203819cd60 con 0x7f2038104750 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2036ffd640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f203819c230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f202c02fe10 tx=0x7f202c02fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:25.672 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f203c9bc640 1 -- 10.20.193.121:0/906667053 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f202c004420 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f203c9bc640 1 -- 10.20.193.121:0/906667053 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f202c038c00 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20381a1770 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f203c9bc640 1 -- 10.20.193.121:0/906667053 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f202c041ae0 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20381a1cf0 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.667+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2038104b50 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.668+0000 7f203c9bc640 1 -- 10.20.193.121:0/906667053 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f202c038770 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.669+0000 7f203c9bc640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f200c0774c0 0x7f200c079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.669+0000 7f203c9bc640 1 -- 10.20.193.121:0/906667053 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 2090+0+0 (secure 0 0 0) 0x7f202c03e070 con 0x7f2038104750 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.669+0000 7f20367fc640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f200c0774c0 0x7f200c079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:25.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.669+0000 7f20367fc640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f200c0774c0 0x7f200c079980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f203819d6e0 tx=0x7f2020005d70 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:25.674 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.670+0000 7f203c9bc640 1 -- 10.20.193.121:0/906667053 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f202c035320 con 0x7f2038104750 2026-02-13T10:38:25.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:25.759+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f20380630c0 con 0x7f200c0774c0 2026-02-13T10:38:26.049 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.049 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.049 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: pgmap v18: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:26.049 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='client.14362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:26.049 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:26.049 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: Detected new or changed devices on trial121 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: Adjusting osd_memory_target on trial121 to 39399M 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:26.050 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:26 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: pgmap v18: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='client.14362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: Detected new or changed devices on trial121 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: Adjusting osd_memory_target on trial121 to 39399M 2026-02-13T10:38:26.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:26.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:26.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:26.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:26 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:27.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:27 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3573811866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aad7001-dd1d-4ab3-bee7-17801f2b1563"}]: dispatch 2026-02-13T10:38:27.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:27 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3573811866' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aad7001-dd1d-4ab3-bee7-17801f2b1563"}]': finished 2026-02-13T10:38:27.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:27 trial163 ceph-mon[24184]: osdmap e11: 3 total, 1 up, 3 in 2026-02-13T10:38:27.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:27 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:27.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:27 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:27.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:27 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3071889653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:27.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:27 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3573811866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aad7001-dd1d-4ab3-bee7-17801f2b1563"}]: dispatch 2026-02-13T10:38:27.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:27 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3573811866' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aad7001-dd1d-4ab3-bee7-17801f2b1563"}]': finished 2026-02-13T10:38:27.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:27 trial121 ceph-mon[19159]: osdmap e11: 3 total, 1 up, 3 in 2026-02-13T10:38:27.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:27 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:27.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:27 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:27.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:27 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3071889653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:28.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:28 trial163 ceph-mon[24184]: pgmap v20: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:28.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:28 trial121 ceph-mon[19159]: pgmap v20: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:29.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:29 trial121 ceph-mon[19159]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2026-02-13T10:38:29.958 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:29 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1[47880]: 2026-02-13T10:38:29.547+0000 7f59fa7f4740 -1 osd.1 0 log_to_monitors true 2026-02-13T10:38:30.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:29 trial163 ceph-mon[24184]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2026-02-13T10:38:30.958 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:30 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1[47880]: 2026-02-13T10:38:30.605+0000 7f59f5f52640 -1 osd.1 0 waiting for initial osdmap 2026-02-13T10:38:30.958 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:30 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1[47880]: 2026-02-13T10:38:30.607+0000 7f59f258d640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: pgmap v21: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: osdmap e12: 3 total, 1 up, 3 in 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial121", "root=default"]}]: dispatch 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:30.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:30 trial121 ceph-mon[19159]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial121", "root=default"]}]': finished 2026-02-13T10:38:31.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: pgmap v21: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:31.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-13T10:38:31.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: osdmap e12: 3 total, 1 up, 3 in 2026-02-13T10:38:31.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial121", "root=default"]}]: dispatch 2026-02-13T10:38:31.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:31.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:31.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:31.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:30 trial163 ceph-mon[24184]: from='osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial121", "root=default"]}]': finished 2026-02-13T10:38:31.765 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:31 trial121 ceph-mon[19159]: osdmap e13: 3 total, 1 up, 3 in 2026-02-13T10:38:31.765 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:31.765 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:31.765 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:31.941 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:31 trial163 ceph-mon[24184]: osdmap e13: 3 total, 1 up, 3 in 2026-02-13T10:38:31.941 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:31.941 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:31.941 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:32.860 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: purged_snaps scrub starts 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: purged_snaps scrub ok 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: pgmap v24: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] boot 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: osdmap e14: 3 total, 2 up, 3 in 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2026-02-13T10:38:32.861 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:33.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: purged_snaps scrub starts 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: purged_snaps scrub ok 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: pgmap v24: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: osd.1 [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] boot 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: osdmap e14: 3 total, 2 up, 3 in 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2026-02-13T10:38:33.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:33.860 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:33 trial121 ceph-mon[19159]: Deploying daemon osd.2 on trial121 2026-02-13T10:38:34.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:33 trial163 ceph-mon[24184]: Deploying daemon osd.2 on trial121 2026-02-13T10:38:34.879 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:34 trial121 ceph-mon[19159]: pgmap v26: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:34.879 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:34 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:34.879 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:34 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:34.879 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:34 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:34.979 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.978+0000 7f203c9bc640 1 -- 10.20.193.121:0/906667053 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f20380630c0 con 0x7f200c0774c0 2026-02-13T10:38:34.979 INFO:teuthology.orchestra.run.trial121.stdout:Created osd(s) 2 on host 'trial121' 2026-02-13T10:38:34.981 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f200c0774c0 msgr2=0x7f200c079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:34.981 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f200c0774c0 0x7f200c079980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f203819d6e0 tx=0x7f2020005d70 comp rx=0 tx=0).stop 2026-02-13T10:38:34.981 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 msgr2=0x7f203819c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f203819c230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f202c02fe10 tx=0x7f202c02fe40 comp rx=0 tx=0).stop 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 shutdown_connections 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f200c0774c0 0x7f200c079980 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2038105950 0x7f203819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 --2- 10.20.193.121:0/906667053 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2038104750 0x7f203819c230 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 >> 10.20.193.121:0/906667053 conn(0x7f20380fff00 msgr2=0x7f2038108fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 shutdown_connections 2026-02-13T10:38:34.982 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:38:34.980+0000 7f2037fff640 1 -- 10.20.193.121:0/906667053 wait complete. 2026-02-13T10:38:35.022 DEBUG:teuthology.orchestra.run.trial121:osd.2> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.2.service 2026-02-13T10:38:35.024 INFO:tasks.cephadm:Deploying osd.3 on trial163 with /dev/vg_nvme/lv_4... 2026-02-13T10:38:35.024 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-13T10:38:35.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:34 trial163 ceph-mon[24184]: pgmap v26: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:35.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:34 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:35.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:34 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:35.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:34 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:35.308 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:38:35.952 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:35 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2[54994]: 2026-02-13T10:38:35.893+0000 7f5427e61740 -1 Falling back to public interface 2026-02-13T10:38:36.070 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:38:36.091 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial163:vg_nvme/lv_4 2026-02-13T10:38:36.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: pgmap v27: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: Detected new or changed devices on trial121 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: Adjusting osd_memory_target on trial121 to 26266M 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: pgmap v27: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: Detected new or changed devices on trial121 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: Adjusting osd_memory_target on trial121 to 26266M 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:36.415 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:38:36.525 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- 10.20.193.163:0/2964018135 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 msgr2=0x7f3514103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 --2- 10.20.193.163:0/2964018135 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f3514103320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f35000099b0 tx=0x7f350002f220 comp rx=0 tx=0).stop 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- 10.20.193.163:0/2964018135 shutdown_connections 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 --2- 10.20.193.163:0/2964018135 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3514103860 0x7f3514105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 --2- 10.20.193.163:0/2964018135 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f3514103320 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- 10.20.193.163:0/2964018135 >> 10.20.193.163:0/2964018135 conn(0x7f35140fcd20 msgr2=0x7f35140ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- 10.20.193.163:0/2964018135 shutdown_connections 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- 10.20.193.163:0/2964018135 wait complete. 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 Processor -- start 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- start start 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f35141a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:36.526 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3514103860 0x7f35141a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3514111b10 con 0x7f3514103860 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.522+0000 7f3519efa640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3514111990 con 0x7f3514100f10 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f35137fe640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f35141a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f35137fe640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f35141a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.163:3300/0 says I am v2:10.20.193.163:47766/0 (socket says 10.20.193.163:47766) 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f35137fe640 1 -- 10.20.193.163:0/3911282040 learned_addr learned my addr 10.20.193.163:0/3911282040 (peer_addr_for_me v2:10.20.193.163:0/0) 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f3512ffd640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3514103860 0x7f35141a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f35137fe640 1 -- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3514103860 msgr2=0x7f35141a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f35137fe640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3514103860 0x7f35141a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f35137fe640 1 -- 10.20.193.163:0/3911282040 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35141a14c0 con 0x7f3514100f10 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f35137fe640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f35141a0900 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f350002fc90 tx=0x7f350002fcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f3510ff9640 1 -- 10.20.193.163:0/3911282040 <== mon.1 v2:10.20.193.163:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f3500004420 con 0x7f3514100f10 2026-02-13T10:38:36.527 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35141a5ea0 con 0x7f3514100f10 2026-02-13T10:38:36.528 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f3510ff9640 1 -- 10.20.193.163:0/3911282040 <== mon.1 v2:10.20.193.163:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f3500038c00 con 0x7f3514100f10 2026-02-13T10:38:36.528 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f3510ff9640 1 -- 10.20.193.163:0/3911282040 <== mon.1 v2:10.20.193.163:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f3500041950 con 0x7f3514100f10 2026-02-13T10:38:36.528 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35141a63d0 con 0x7f3514100f10 2026-02-13T10:38:36.528 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.523+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3514101310 con 0x7f3514100f10 2026-02-13T10:38:36.528 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.524+0000 7f3510ff9640 1 -- 10.20.193.163:0/3911282040 <== mon.1 v2:10.20.193.163:3300/0 4 ==== mgrmap(e 18) ==== 100063+0+0 (secure 0 0 0) 0x7f3500041c50 con 0x7f3514100f10 2026-02-13T10:38:36.529 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.524+0000 7f3510ff9640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f34e80774c0 0x7f34e8079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:36.529 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.524+0000 7f3510ff9640 1 -- 10.20.193.163:0/3911282040 <== mon.1 v2:10.20.193.163:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2522+0+0 (secure 0 0 0) 0x7f350003e070 con 0x7f3514100f10 2026-02-13T10:38:36.529 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.525+0000 7f3512ffd640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f34e80774c0 0x7f34e8079980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:36.529 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.525+0000 7f3512ffd640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f34e80774c0 0x7f34e8079980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35141a1db0 tx=0x7f3504009290 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:36.529 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.526+0000 7f3510ff9640 1 -- 10.20.193.163:0/3911282040 <== mon.1 v2:10.20.193.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f350008b890 con 0x7f3514100f10 2026-02-13T10:38:36.614 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:36.613+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f35141041c0 con 0x7f34e80774c0 2026-02-13T10:38:37.137 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:37 trial163 ceph-mon[24184]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:37.137 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:37 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:37.138 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:37 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:37.138 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:37 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:37 trial121 ceph-mon[19159]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:37 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:37 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:37.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:37 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:38.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: from='client.? 10.20.193.163:0/1114304710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d5bd71-44e9-4c3f-a3b6-f15074495bd4"}]: dispatch 2026-02-13T10:38:38.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d5bd71-44e9-4c3f-a3b6-f15074495bd4"}]: dispatch 2026-02-13T10:38:38.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d5bd71-44e9-4c3f-a3b6-f15074495bd4"}]': finished 2026-02-13T10:38:38.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: osdmap e15: 4 total, 2 up, 4 in 2026-02-13T10:38:38.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:38.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:38.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: pgmap v29: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:38.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:38 trial163 ceph-mon[24184]: from='client.? 10.20.193.163:0/403085002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:38.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1114304710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d5bd71-44e9-4c3f-a3b6-f15074495bd4"}]: dispatch 2026-02-13T10:38:38.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d5bd71-44e9-4c3f-a3b6-f15074495bd4"}]: dispatch 2026-02-13T10:38:38.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d5bd71-44e9-4c3f-a3b6-f15074495bd4"}]': finished 2026-02-13T10:38:38.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: osdmap e15: 4 total, 2 up, 4 in 2026-02-13T10:38:38.458 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:38.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:38.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: pgmap v29: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:38.459 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:38 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/403085002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:40.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:40 trial121 ceph-mon[19159]: pgmap v30: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:40.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:40 trial121 ceph-mon[19159]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2026-02-13T10:38:40.709 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:40 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2[54994]: 2026-02-13T10:38:40.402+0000 7f5427e61740 -1 osd.2 0 log_to_monitors true 2026-02-13T10:38:41.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:40 trial163 ceph-mon[24184]: pgmap v30: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:41.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:40 trial163 ceph-mon[24184]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2026-02-13T10:38:41.958 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:41 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2[54994]: 2026-02-13T10:38:41.609+0000 7f54235bf640 -1 osd.2 0 waiting for initial osdmap 2026-02-13T10:38:41.958 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:41 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2[54994]: 2026-02-13T10:38:41.611+0000 7f541fbfa640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: osdmap e16: 4 total, 2 up, 4 in 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial121", "root=default"]}]: dispatch 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial121", "root=default"]}]': finished 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: osdmap e17: 4 total, 2 up, 4 in 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:41.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:41 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: osdmap e16: 4 total, 2 up, 4 in 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial121", "root=default"]}]: dispatch 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: from='osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial121", "root=default"]}]': finished 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: osdmap e17: 4 total, 2 up, 4 in 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:42.006 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:41 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:42.880 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:42 trial163 ceph-mon[24184]: pgmap v32: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:42.880 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:42 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:42.880 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:42 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:42.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:42 trial121 ceph-mon[19159]: pgmap v32: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:42.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:42 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:42.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:42 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: purged_snaps scrub starts 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: purged_snaps scrub ok 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] boot 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: osdmap e18: 4 total, 3 up, 4 in 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2026-02-13T10:38:43.691 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:43.692 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:43 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2026-02-13T10:38:43.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: purged_snaps scrub starts 2026-02-13T10:38:43.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: purged_snaps scrub ok 2026-02-13T10:38:43.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: osd.2 [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] boot 2026-02-13T10:38:43.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: osdmap e18: 4 total, 3 up, 4 in 2026-02-13T10:38:43.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2026-02-13T10:38:43.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:43.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2026-02-13T10:38:43.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:43.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:43 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2026-02-13T10:38:44.738 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:44 trial163 ceph-mon[24184]: Deploying daemon osd.3 on trial163 2026-02-13T10:38:44.738 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:44 trial163 ceph-mon[24184]: pgmap v35: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:44.738 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:44 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-13T10:38:44.738 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:44 trial163 ceph-mon[24184]: osdmap e19: 4 total, 3 up, 4 in 2026-02-13T10:38:44.739 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:44 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:44.739 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:44 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2026-02-13T10:38:44.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:44 trial121 ceph-mon[19159]: Deploying daemon osd.3 on trial163 2026-02-13T10:38:44.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:44 trial121 ceph-mon[19159]: pgmap v35: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T10:38:44.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:44 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-13T10:38:44.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:44 trial121 ceph-mon[19159]: osdmap e19: 4 total, 3 up, 4 in 2026-02-13T10:38:44.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:44 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:44.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:44 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2026-02-13T10:38:45.625 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58573]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-13T10:38:45.625 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58573]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.625 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58573]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.625 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58573]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.625 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-13T10:38:45.626 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.626 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58577]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.626 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58577]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.626 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2026-02-13T10:38:45.626 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.626 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58597]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.626 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58597]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.627 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1 2026-02-13T10:38:45.627 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.627 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58601]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.627 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58601]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.627 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-13T10:38:45.627 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.627 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58581]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58581]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58585]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58585]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58585]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58589]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58589]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58589]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.628 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58589]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.629 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58593]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1 2026-02-13T10:38:45.629 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.629 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58593]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.629 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:38:45 trial121 sudo[58593]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30791]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30791]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30791]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30791]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme0n1 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30817]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T10:38:45.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 sudo[30817]: pam_unix(sudo:session): session closed for user root 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: osdmap e20: 4 total, 3 up, 4 in 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:38:45.792 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.793 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:45 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:45.854 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.852+0000 7f3510ff9640 1 -- 10.20.193.163:0/3911282040 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f35141041c0 con 0x7f34e80774c0 2026-02-13T10:38:45.854 INFO:teuthology.orchestra.run.trial163.stdout:Created osd(s) 3 on host 'trial163' 2026-02-13T10:38:45.856 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f34e80774c0 msgr2=0x7f34e8079980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:45.856 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f34e80774c0 0x7f34e8079980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35141a1db0 tx=0x7f3504009290 comp rx=0 tx=0).stop 2026-02-13T10:38:45.856 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 msgr2=0x7f35141a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:45.856 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f35141a0900 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f350002fc90 tx=0x7f350002fcc0 comp rx=0 tx=0).stop 2026-02-13T10:38:45.856 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 shutdown_connections 2026-02-13T10:38:45.856 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f34e80774c0 0x7f34e8079980 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:45.857 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3514103860 0x7f35141a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:45.857 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 --2- 10.20.193.163:0/3911282040 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3514100f10 0x7f35141a0900 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:45.857 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 >> 10.20.193.163:0/3911282040 conn(0x7f35140fcd20 msgr2=0x7f35140ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:45.857 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 shutdown_connections 2026-02-13T10:38:45.857 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:45.854+0000 7f3519efa640 1 -- 10.20.193.163:0/3911282040 wait complete. 2026-02-13T10:38:45.909 DEBUG:teuthology.orchestra.run.trial163:osd.3> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.3.service 2026-02-13T10:38:45.911 INFO:tasks.cephadm:Deploying osd.4 on trial163 with /dev/vg_nvme/lv_3... 2026-02-13T10:38:45.911 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-13T10:38:45.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-13T10:38:45.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: osdmap e20: 4 total, 3 up, 4 in 2026-02-13T10:38:45.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:45.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:45.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial121"}]: dispatch 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mon metadata", "id": "trial163"}]: dispatch 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T10:38:45.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:45.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:45.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:45.960 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:45 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:38:46.244 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:38:46.828 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: pgmap v38: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:46.828 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: osdmap e21: 4 total, 3 up, 4 in 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: mgrmap e19: trial121.tkoihk(active, since 60s), standbys: trial163.xfyaeo 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:46.829 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:46.830 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:46 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: pgmap v38: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: osdmap e21: 4 total, 3 up, 4 in 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: mgrmap e19: trial121.tkoihk(active, since 60s), standbys: trial163.xfyaeo 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:46.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:46 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:47.228 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:38:46 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3[30217]: 2026-02-13T10:38:46.826+0000 7f0c96d67740 -1 Falling back to public interface 2026-02-13T10:38:47.279 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:38:47.297 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial163:vg_nvme/lv_3 2026-02-13T10:38:47.587 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- 10.20.193.163:0/381528267 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1026d0 msgr2=0x7ff9cc102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 --2- 10.20.193.163:0/381528267 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1026d0 0x7ff9cc102ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80099b0 tx=0x7ff9b802f220 comp rx=0 tx=0).stop 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- 10.20.193.163:0/381528267 shutdown_connections 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 --2- 10.20.193.163:0/381528267 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff9cc1038d0 0x7ff9cc103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 --2- 10.20.193.163:0/381528267 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1026d0 0x7ff9cc102ad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- 10.20.193.163:0/381528267 >> 10.20.193.163:0/381528267 conn(0x7ff9cc0fde60 msgr2=0x7ff9cc1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- 10.20.193.163:0/381528267 shutdown_connections 2026-02-13T10:38:47.689 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- 10.20.193.163:0/381528267 wait complete. 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 Processor -- start 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- start start 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff9cc1026d0 0x7ff9cc19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1038d0 0x7ff9cc19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10b3c0 con 0x7ff9cc1026d0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9d208a640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc10b240 con 0x7ff9cc1038d0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9caffd640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1038d0 0x7ff9cc19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9caffd640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1038d0 0x7ff9cc19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.163:3300/0 says I am v2:10.20.193.163:37572/0 (socket says 10.20.193.163:37572) 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.687+0000 7ff9caffd640 1 -- 10.20.193.163:0/2770637187 learned_addr learned my addr 10.20.193.163:0/2770637187 (peer_addr_for_me v2:10.20.193.163:0/0) 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9cb7fe640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff9cc1026d0 0x7ff9cc19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9caffd640 1 -- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff9cc1026d0 msgr2=0x7ff9cc19c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9caffd640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff9cc1026d0 0x7ff9cc19c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9caffd640 1 -- 10.20.193.163:0/2770637187 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc19cff0 con 0x7ff9cc1038d0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9caffd640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1038d0 0x7ff9cc19c970 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00e9b0 tx=0x7ff9bc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9c8ff9640 1 -- 10.20.193.163:0/2770637187 <== mon.1 v2:10.20.193.163:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ff9bc00cbe0 con 0x7ff9cc1038d0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9c8ff9640 1 -- 10.20.193.163:0/2770637187 <== mon.1 v2:10.20.193.163:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7ff9bc004560 con 0x7ff9cc1038d0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc1a1a30 con 0x7ff9cc1038d0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9c8ff9640 1 -- 10.20.193.163:0/2770637187 <== mon.1 v2:10.20.193.163:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ff9bc010430 con 0x7ff9cc1038d0 2026-02-13T10:38:47.690 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1a2040 con 0x7ff9cc1038d0 2026-02-13T10:38:47.691 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.688+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc102ad0 con 0x7ff9cc1038d0 2026-02-13T10:38:47.691 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.689+0000 7ff9c8ff9640 1 -- 10.20.193.163:0/2770637187 <== mon.1 v2:10.20.193.163:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7ff9bc0040d0 con 0x7ff9cc1038d0 2026-02-13T10:38:47.691 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.689+0000 7ff9c8ff9640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9a00775a0 0x7ff9a0079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:47.692 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.690+0000 7ff9c8ff9640 1 -- 10.20.193.163:0/2770637187 <== mon.1 v2:10.20.193.163:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3340+0+0 (secure 0 0 0) 0x7ff9bc099260 con 0x7ff9cc1038d0 2026-02-13T10:38:47.692 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.690+0000 7ff9cb7fe640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9a00775a0 0x7ff9a0079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:47.692 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.690+0000 7ff9cb7fe640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9a00775a0 0x7ff9a0079a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005e90 tx=0x7ff9b803a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:47.694 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.692+0000 7ff9c8ff9640 1 -- 10.20.193.163:0/2770637187 <== mon.1 v2:10.20.193.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff9bc066c00 con 0x7ff9cc1038d0 2026-02-13T10:38:47.783 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:47.781+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff9cc0630c0 con 0x7ff9a00775a0 2026-02-13T10:38:47.957 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:47 trial121 ceph-mon[19159]: Detected new or changed devices on trial163 2026-02-13T10:38:47.957 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:47 trial121 ceph-mon[19159]: Adjusting osd_memory_target on trial163 to 81871M 2026-02-13T10:38:47.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:47 trial121 ceph-mon[19159]: osdmap e22: 4 total, 3 up, 4 in 2026-02-13T10:38:47.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:47 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:47.966 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:47 trial163 ceph-mon[24184]: Detected new or changed devices on trial163 2026-02-13T10:38:47.966 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:47 trial163 ceph-mon[24184]: Adjusting osd_memory_target on trial163 to 81871M 2026-02-13T10:38:47.966 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:47 trial163 ceph-mon[24184]: osdmap e22: 4 total, 3 up, 4 in 2026-02-13T10:38:47.966 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:47 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: pgmap v41: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/469789474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d965c3c-65dd-4fd9-a1ee-996241c8d2e9"}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d965c3c-65dd-4fd9-a1ee-996241c8d2e9"}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d965c3c-65dd-4fd9-a1ee-996241c8d2e9"}]': finished 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: osdmap e23: 5 total, 3 up, 5 in 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:48.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:48 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:49.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: pgmap v41: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:49.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='client.? 10.20.193.163:0/469789474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d965c3c-65dd-4fd9-a1ee-996241c8d2e9"}]: dispatch 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d965c3c-65dd-4fd9-a1ee-996241c8d2e9"}]: dispatch 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d965c3c-65dd-4fd9-a1ee-996241c8d2e9"}]': finished 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: osdmap e23: 5 total, 3 up, 5 in 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:49.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:48 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:49.957 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:49 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1992059391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:50.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:49 trial163 ceph-mon[24184]: from='client.? 10.20.193.163:0/1992059391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:38:50.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:50 trial121 ceph-mon[19159]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:51.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:50 trial163 ceph-mon[24184]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:51.635 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:38:51 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3[30217]: 2026-02-13T10:38:51.352+0000 7f0c96d67740 -1 osd.3 0 log_to_monitors true 2026-02-13T10:38:51.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:51 trial121 ceph-mon[19159]: from='osd.3 [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2026-02-13T10:38:51.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:51 trial121 ceph-mon[19159]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2026-02-13T10:38:52.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:51 trial163 ceph-mon[24184]: from='osd.3 [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2026-02-13T10:38:52.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:51 trial163 ceph-mon[24184]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2026-02-13T10:38:52.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:52 trial121 ceph-mon[19159]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:52.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:52 trial121 ceph-mon[19159]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-13T10:38:52.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:52 trial121 ceph-mon[19159]: osdmap e24: 5 total, 3 up, 5 in 2026-02-13T10:38:52.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:52 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:52.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:52 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:52.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:52 trial121 ceph-mon[19159]: from='osd.3 [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:38:52.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:52 trial121 ceph-mon[19159]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:38:53.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:52 trial163 ceph-mon[24184]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:53.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:52 trial163 ceph-mon[24184]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-13T10:38:53.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:52 trial163 ceph-mon[24184]: osdmap e24: 5 total, 3 up, 5 in 2026-02-13T10:38:53.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:52 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:53.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:52 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:53.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:52 trial163 ceph-mon[24184]: from='osd.3 [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:38:53.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:52 trial163 ceph-mon[24184]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:38:53.041 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:38:52 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3[30217]: 2026-02-13T10:38:52.643+0000 7f0c924c5640 -1 osd.3 0 waiting for initial osdmap 2026-02-13T10:38:53.041 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:38:52 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3[30217]: 2026-02-13T10:38:52.646+0000 7f0c8eb00640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T10:38:53.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:53 trial163 ceph-mon[24184]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial163", "root=default"]}]': finished 2026-02-13T10:38:53.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:53 trial163 ceph-mon[24184]: osdmap e25: 5 total, 3 up, 5 in 2026-02-13T10:38:53.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:53 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:53.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:53 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:53.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:53 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:53.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:53 trial121 ceph-mon[19159]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial163", "root=default"]}]': finished 2026-02-13T10:38:53.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:53 trial121 ceph-mon[19159]: osdmap e25: 5 total, 3 up, 5 in 2026-02-13T10:38:53.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:53 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:53.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:53 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:53.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:53 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:54.873 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: purged_snaps scrub starts 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: purged_snaps scrub ok 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: osd.3 [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] boot 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: osdmap e26: 5 total, 4 up, 5 in 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2026-02-13T10:38:54.874 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:54 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: purged_snaps scrub starts 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: purged_snaps scrub ok 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: osd.3 [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] boot 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: osdmap e26: 5 total, 4 up, 5 in 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2026-02-13T10:38:54.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:54 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:55.912 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:55 trial163 ceph-mon[24184]: Deploying daemon osd.4 on trial163 2026-02-13T10:38:55.912 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:55 trial163 ceph-mon[24184]: osdmap e27: 5 total, 4 up, 5 in 2026-02-13T10:38:55.912 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:55 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:55.912 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:55 trial163 ceph-mon[24184]: osdmap e28: 5 total, 4 up, 5 in 2026-02-13T10:38:55.912 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:55 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:55.957 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:55 trial121 ceph-mon[19159]: Deploying daemon osd.4 on trial163 2026-02-13T10:38:55.957 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:55 trial121 ceph-mon[19159]: osdmap e27: 5 total, 4 up, 5 in 2026-02-13T10:38:55.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:55 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:55.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:55 trial121 ceph-mon[19159]: osdmap e28: 5 total, 4 up, 5 in 2026-02-13T10:38:55.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:55 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:38:56.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:56 trial121 ceph-mon[19159]: pgmap v50: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T10:38:56.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:56 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:56.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:56 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:56.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:56 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.013 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:56 trial163 ceph-mon[24184]: pgmap v50: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T10:38:57.013 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:56 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:57.013 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:56 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.013 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:56 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.061 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.059+0000 7ff9c8ff9640 1 -- 10.20.193.163:0/2770637187 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7ff9cc0630c0 con 0x7ff9a00775a0 2026-02-13T10:38:57.061 INFO:teuthology.orchestra.run.trial163.stdout:Created osd(s) 4 on host 'trial163' 2026-02-13T10:38:57.063 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9a00775a0 msgr2=0x7ff9a0079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:57.063 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9a00775a0 0x7ff9a0079a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005e90 tx=0x7ff9b803a040 comp rx=0 tx=0).stop 2026-02-13T10:38:57.063 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1038d0 msgr2=0x7ff9cc19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1038d0 0x7ff9cc19c970 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00e9b0 tx=0x7ff9bc00ee80 comp rx=0 tx=0).stop 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 shutdown_connections 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9a00775a0 0x7ff9a0079a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff9cc1038d0 0x7ff9cc19c970 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 --2- 10.20.193.163:0/2770637187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff9cc1026d0 0x7ff9cc19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 >> 10.20.193.163:0/2770637187 conn(0x7ff9cc0fde60 msgr2=0x7ff9cc104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 shutdown_connections 2026-02-13T10:38:57.064 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:57.061+0000 7ff9d208a640 1 -- 10.20.193.163:0/2770637187 wait complete. 2026-02-13T10:38:57.100 DEBUG:teuthology.orchestra.run.trial163:osd.4> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.4.service 2026-02-13T10:38:57.102 INFO:tasks.cephadm:Deploying osd.5 on trial163 with /dev/vg_nvme/lv_2... 2026-02-13T10:38:57.102 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-13T10:38:57.430 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.955 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:57 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:57.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:57 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:58.290 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:38:57 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:38:57.995+0000 7fdaa0063740 -1 Falling back to public interface 2026-02-13T10:38:58.462 INFO:teuthology.orchestra.run.trial163.stdout: 2026-02-13T10:38:58.480 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph orch daemon add osd trial163:vg_nvme/lv_2 2026-02-13T10:38:58.765 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 -- 10.20.193.163:0/1338578682 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 msgr2=0x7f9e98109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 --2- 10.20.193.163:0/1338578682 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e98109230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9e900099b0 tx=0x7f9e9002f220 comp rx=0 tx=0).stop 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 -- 10.20.193.163:0/1338578682 shutdown_connections 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 --2- 10.20.193.163:0/1338578682 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e98109f90 0x7f9e9810a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 --2- 10.20.193.163:0/1338578682 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e98109230 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 -- 10.20.193.163:0/1338578682 >> 10.20.193.163:0/1338578682 conn(0x7f9e98077de0 msgr2=0x7f9e9807a220 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 -- 10.20.193.163:0/1338578682 shutdown_connections 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 -- 10.20.193.163:0/1338578682 wait complete. 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 Processor -- start 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 -- start start 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.875+0000 7f9ea0882640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e981a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9ea0882640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e98109f90 0x7f9e981a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9ea0882640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98111aa0 con 0x7f9e98109f90 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9ea0882640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98111920 con 0x7f9e98108e30 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9e5f7640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e981a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9e5f7640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e981a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.163:3300/0 says I am v2:10.20.193.163:59882/0 (socket says 10.20.193.163:59882) 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9e5f7640 1 -- 10.20.193.163:0/3736535022 learned_addr learned my addr 10.20.193.163:0/3736535022 (peer_addr_for_me v2:10.20.193.163:0/0) 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9ddf6640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e98109f90 0x7f9e981a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9e5f7640 1 -- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e98109f90 msgr2=0x7f9e981a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9e5f7640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e98109f90 0x7f9e981a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9e5f7640 1 -- 10.20.193.163:0/3736535022 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e981a14f0 con 0x7f9e98108e30 2026-02-13T10:38:58.877 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e9e5f7640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e981a0930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e9002fc90 tx=0x7f9e9002fcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:58.878 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e837fe640 1 -- 10.20.193.163:0/3736535022 <== mon.1 v2:10.20.193.163:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9e90004420 con 0x7f9e98108e30 2026-02-13T10:38:58.878 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e837fe640 1 -- 10.20.193.163:0/3736535022 <== mon.1 v2:10.20.193.163:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f9e90038c00 con 0x7f9e98108e30 2026-02-13T10:38:58.878 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981a5ed0 con 0x7f9e98108e30 2026-02-13T10:38:58.878 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9e837fe640 1 -- 10.20.193.163:0/3736535022 <== mon.1 v2:10.20.193.163:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9e90041950 con 0x7f9e98108e30 2026-02-13T10:38:58.878 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e981a6400 con 0x7f9e98108e30 2026-02-13T10:38:58.878 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.876+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e98109230 con 0x7f9e98108e30 2026-02-13T10:38:58.878 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.877+0000 7f9e837fe640 1 -- 10.20.193.163:0/3736535022 <== mon.1 v2:10.20.193.163:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f9e90041c00 con 0x7f9e98108e30 2026-02-13T10:38:58.879 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.878+0000 7f9e837fe640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9e740775a0 0x7f9e74079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:38:58.879 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.878+0000 7f9e837fe640 1 -- 10.20.193.163:0/3736535022 <== mon.1 v2:10.20.193.163:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3880+0+0 (secure 0 0 0) 0x7f9e9003e070 con 0x7f9e98108e30 2026-02-13T10:38:58.879 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.878+0000 7f9e9ddf6640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9e740775a0 0x7f9e74079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:38:58.879 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.878+0000 7f9e9ddf6640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9e740775a0 0x7f9e74079a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e981a1de0 tx=0x7f9e84009290 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:38:58.880 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.880+0000 7f9e837fe640 1 -- 10.20.193.163:0/3736535022 <== mon.1 v2:10.20.193.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9e90049340 con 0x7f9e98108e30 2026-02-13T10:38:58.968 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:38:58.967+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9e980630c0 con 0x7f9e740775a0 2026-02-13T10:38:58.968 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: pgmap v52: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T10:38:58.968 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: Detected new or changed devices on trial163 2026-02-13T10:38:58.968 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:58.968 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:58.968 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:58.968 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:58.968 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: Adjusting osd_memory_target on trial163 to 40935M 2026-02-13T10:38:58.969 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:58.969 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:58.969 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:58.969 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:58 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: pgmap v52: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: Detected new or changed devices on trial163 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: Adjusting osd_memory_target on trial163 to 40935M 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:38:59.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:58 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='client.? 10.20.193.163:0/88203874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e8a5b3-c3d6-4c49-97ce-8641644d24cb"}]: dispatch 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e8a5b3-c3d6-4c49-97ce-8641644d24cb"}]: dispatch 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5e8a5b3-c3d6-4c49-97ce-8641644d24cb"}]': finished 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: osdmap e29: 6 total, 4 up, 6 in 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:00.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:38:59 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:00.207 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/88203874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e8a5b3-c3d6-4c49-97ce-8641644d24cb"}]: dispatch 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5e8a5b3-c3d6-4c49-97ce-8641644d24cb"}]: dispatch 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5e8a5b3-c3d6-4c49-97ce-8641644d24cb"}]': finished 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: osdmap e29: 6 total, 4 up, 6 in 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:00.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:38:59 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:01.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:00 trial121 ceph-mon[19159]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:01.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:00 trial121 ceph-mon[19159]: pgmap v53: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-13T10:39:01.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:00 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/1723857011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:39:01.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:00 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:39:01.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:00 trial163 ceph-mon[24184]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:01.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:00 trial163 ceph-mon[24184]: pgmap v53: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-13T10:39:01.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:00 trial163 ceph-mon[24184]: from='client.? 10.20.193.163:0/1723857011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2026-02-13T10:39:01.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:00 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:39:02.799 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:02 trial163 ceph-mon[24184]: pgmap v55: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 83 KiB/s, 0 objects/s recovering 2026-02-13T10:39:02.799 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:02 trial163 ceph-mon[24184]: from='osd.4 [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2026-02-13T10:39:02.799 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:02 trial163 ceph-mon[24184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2026-02-13T10:39:02.799 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:39:02 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:39:02.547+0000 7fdaa0063740 -1 osd.4 0 log_to_monitors true 2026-02-13T10:39:03.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:02 trial121 ceph-mon[19159]: pgmap v55: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 83 KiB/s, 0 objects/s recovering 2026-02-13T10:39:03.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:02 trial121 ceph-mon[19159]: from='osd.4 [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2026-02-13T10:39:03.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:02 trial121 ceph-mon[19159]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2026-02-13T10:39:04.040 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:39:03 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:39:03.806+0000 7fda9b7c1640 -1 osd.4 0 waiting for initial osdmap 2026-02-13T10:39:04.040 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:39:03 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:39:03.809+0000 7fda97dfc640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T10:39:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:03 trial163 ceph-mon[24184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-13T10:39:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:03 trial163 ceph-mon[24184]: osdmap e30: 6 total, 4 up, 6 in 2026-02-13T10:39:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:03 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:03 trial163 ceph-mon[24184]: from='osd.4 [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:04.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:03 trial163 ceph-mon[24184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:04.207 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:03 trial121 ceph-mon[19159]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-13T10:39:04.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:03 trial121 ceph-mon[19159]: osdmap e30: 6 total, 4 up, 6 in 2026-02-13T10:39:04.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:04.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:03 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:04.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:03 trial121 ceph-mon[19159]: from='osd.4 [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:04.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:03 trial121 ceph-mon[19159]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:04.982 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:04 trial163 ceph-mon[24184]: pgmap v57: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 73 KiB/s, 0 objects/s recovering 2026-02-13T10:39:04.982 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:04 trial163 ceph-mon[24184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial163", "root=default"]}]': finished 2026-02-13T10:39:04.982 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:04 trial163 ceph-mon[24184]: osdmap e31: 6 total, 4 up, 6 in 2026-02-13T10:39:04.982 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:04 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:04.982 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:04 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:04.982 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:04 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:05.207 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:04 trial121 ceph-mon[19159]: pgmap v57: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 73 KiB/s, 0 objects/s recovering 2026-02-13T10:39:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:04 trial121 ceph-mon[19159]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial163", "root=default"]}]': finished 2026-02-13T10:39:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:04 trial121 ceph-mon[19159]: osdmap e31: 6 total, 4 up, 6 in 2026-02-13T10:39:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:04 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:04 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:05.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:04 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: purged_snaps scrub starts 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: purged_snaps scrub ok 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: osd.4 [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] boot 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: osdmap e32: 6 total, 5 up, 6 in 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2026-02-13T10:39:06.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:05 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: purged_snaps scrub starts 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: purged_snaps scrub ok 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: osd.4 [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] boot 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: osdmap e32: 6 total, 5 up, 6 in 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2026-02-13T10:39:06.109 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:05 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:07.037 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:06 trial163 ceph-mon[24184]: Deploying daemon osd.5 on trial163 2026-02-13T10:39:07.037 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:06 trial163 ceph-mon[24184]: pgmap v60: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T10:39:07.038 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:06 trial163 ceph-mon[24184]: osdmap e33: 6 total, 5 up, 6 in 2026-02-13T10:39:07.038 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:06 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:07.207 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:06 trial121 ceph-mon[19159]: Deploying daemon osd.5 on trial163 2026-02-13T10:39:07.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:06 trial121 ceph-mon[19159]: pgmap v60: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T10:39:07.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:06 trial121 ceph-mon[19159]: osdmap e33: 6 total, 5 up, 6 in 2026-02-13T10:39:07.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:06 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:08.070 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:07 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:08.071 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:07 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:08.071 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:07 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:08.095 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.093+0000 7f9e837fe640 1 -- 10.20.193.163:0/3736535022 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f9e980630c0 con 0x7f9e740775a0 2026-02-13T10:39:08.095 INFO:teuthology.orchestra.run.trial163.stdout:Created osd(s) 5 on host 'trial163' 2026-02-13T10:39:08.097 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9e740775a0 msgr2=0x7f9e74079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:08.097 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9e740775a0 0x7f9e74079a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e981a1de0 tx=0x7f9e84009290 comp rx=0 tx=0).stop 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 msgr2=0x7f9e981a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e981a0930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e9002fc90 tx=0x7f9e9002fcc0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 shutdown_connections 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9e740775a0 0x7f9e74079a60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e98109f90 0x7f9e981a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 --2- 10.20.193.163:0/3736535022 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e98108e30 0x7f9e981a0930 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.095+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 >> 10.20.193.163:0/3736535022 conn(0x7f9e98077de0 msgr2=0x7f9e9810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.096+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 shutdown_connections 2026-02-13T10:39:08.098 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:08.096+0000 7f9ea0882640 1 -- 10.20.193.163:0/3736535022 wait complete. 2026-02-13T10:39:08.136 DEBUG:teuthology.orchestra.run.trial163:osd.5> sudo journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.5.service 2026-02-13T10:39:08.138 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2026-02-13T10:39:08.138 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T10:39:08.162 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:07 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:08.162 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:07 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:08.162 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:07 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:08.429 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:08.541 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- 10.20.193.121:0/400244812 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 msgr2=0x7f9198105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 --2- 10.20.193.121:0/400244812 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f9198105fa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f91880099b0 tx=0x7f918802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- 10.20.193.121:0/400244812 shutdown_connections 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 --2- 10.20.193.121:0/400244812 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f9198105fa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 --2- 10.20.193.121:0/400244812 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9198104920 0x7f9198104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- 10.20.193.121:0/400244812 >> 10.20.193.121:0/400244812 conn(0x7f91981000f0 msgr2=0x7f9198102530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- 10.20.193.121:0/400244812 shutdown_connections 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- 10.20.193.121:0/400244812 wait complete. 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 Processor -- start 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- start start 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9198104920 0x7f919819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f919819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f919810d5f0 con 0x7f9198105b20 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.540+0000 7f919e874640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f919810d470 con 0x7f9198104920 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91977fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f919819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91977fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f919819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:35650/0 (socket says 10.20.193.121:35650) 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91977fe640 1 -- 10.20.193.121:0/1276718908 learned_addr learned my addr 10.20.193.121:0/1276718908 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f9197fff640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9198104920 0x7f919819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91977fe640 1 -- 10.20.193.121:0/1276718908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9198104920 msgr2=0x7f919819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91977fe640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9198104920 0x7f919819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91977fe640 1 -- 10.20.193.121:0/1276718908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919819cf70 con 0x7f9198105b20 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91977fe640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f919819c980 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9188009ae0 tx=0x7f9188002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:08.542 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91957fa640 1 -- 10.20.193.121:0/1276718908 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9188004420 con 0x7f9198105b20 2026-02-13T10:39:08.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91957fa640 1 -- 10.20.193.121:0/1276718908 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f9188038c00 con 0x7f9198105b20 2026-02-13T10:39:08.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91981a1980 con 0x7f9198105b20 2026-02-13T10:39:08.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f91957fa640 1 -- 10.20.193.121:0/1276718908 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9188041b10 con 0x7f9198105b20 2026-02-13T10:39:08.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91981a1f00 con 0x7f9198105b20 2026-02-13T10:39:08.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.541+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9198104d20 con 0x7f9198105b20 2026-02-13T10:39:08.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.542+0000 7f91957fa640 1 -- 10.20.193.121:0/1276718908 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f9188038770 con 0x7f9198105b20 2026-02-13T10:39:08.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.542+0000 7f91957fa640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f916c0775a0 0x7f916c079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:08.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.543+0000 7f91957fa640 1 -- 10.20.193.121:0/1276718908 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4312+0+0 (secure 0 0 0) 0x7f918803e070 con 0x7f9198105b20 2026-02-13T10:39:08.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.543+0000 7f9197fff640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f916c0775a0 0x7f916c079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:08.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.543+0000 7f9197fff640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f916c0775a0 0x7f916c079a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f91840059c0 tx=0x7f9184005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:08.546 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.545+0000 7f91957fa640 1 -- 10.20.193.121:0/1276718908 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f918808b7c0 con 0x7f9198105b20 2026-02-13T10:39:08.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.630+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f91981a28c0 con 0x7f9198105b20 2026-02-13T10:39:08.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.630+0000 7f91957fa640 1 -- 10.20.193.121:0/1276718908 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v33) ==== 74+0+130 (secure 0 0 0) 0x7f9188090670 con 0x7f9198105b20 2026-02-13T10:39:08.631 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f916c0775a0 msgr2=0x7f916c079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f916c0775a0 0x7f916c079a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f91840059c0 tx=0x7f9184005950 comp rx=0 tx=0).stop 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 msgr2=0x7f919819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f919819c980 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9188009ae0 tx=0x7f9188002e20 comp rx=0 tx=0).stop 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 shutdown_connections 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f916c0775a0 0x7f916c079a60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9198105b20 0x7f919819c980 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 --2- 10.20.193.121:0/1276718908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9198104920 0x7f919819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:08.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 >> 10.20.193.121:0/1276718908 conn(0x7f91981000f0 msgr2=0x7f9198102530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:08.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 shutdown_connections 2026-02-13T10:39:08.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:08.631+0000 7f919e874640 1 -- 10.20.193.121:0/1276718908 wait complete. 2026-02-13T10:39:08.670 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":33,"num_osds":6,"num_up_osds":5,"osd_up_since":1770979144,"num_in_osds":6,"osd_in_since":1770979139,"num_remapped_pgs":0} 2026-02-13T10:39:09.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: pgmap v62: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1276718908' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:09.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:08 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: pgmap v62: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:09.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:09.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:09.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:09.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1276718908' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:09.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:08 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.374 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:39:09 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:39:09.088+0000 7f0239ea0740 -1 Falling back to public interface 2026-02-13T10:39:09.670 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: Detected new or changed devices on trial163 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: Adjusting osd_memory_target on trial163 to 27290M 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:09.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:09 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:09.958 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:10.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- 10.20.193.121:0/3836660598 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 msgr2=0x7ff980105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:10.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 --2- 10.20.193.121:0/3836660598 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff980105fc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff9680099b0 tx=0x7ff96802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- 10.20.193.121:0/3836660598 shutdown_connections 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 --2- 10.20.193.121:0/3836660598 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff980105fc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 --2- 10.20.193.121:0/3836660598 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff980104940 0x7ff980104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- 10.20.193.121:0/3836660598 >> 10.20.193.121:0/3836660598 conn(0x7ff9801000f0 msgr2=0x7ff980102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- 10.20.193.121:0/3836660598 shutdown_connections 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- 10.20.193.121:0/3836660598 wait complete. 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 Processor -- start 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- start start 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff980104940 0x7ff98019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:10.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff98019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff98010d6b0 con 0x7ff980105b40 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff9858cb640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff98010d530 con 0x7ff980104940 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff97e7fc640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff98019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff97e7fc640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff98019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:35670/0 (socket says 10.20.193.121:35670) 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff97e7fc640 1 -- 10.20.193.121:0/2753841909 learned_addr learned my addr 10.20.193.121:0/2753841909 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.064+0000 7ff97effd640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff980104940 0x7ff98019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff97e7fc640 1 -- 10.20.193.121:0/2753841909 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff980104940 msgr2=0x7ff98019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff97e7fc640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff980104940 0x7ff98019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff97e7fc640 1 -- 10.20.193.121:0/2753841909 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98019d0e0 con 0x7ff980105b40 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff97e7fc640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff98019ca60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff968009ae0 tx=0x7ff968002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff9848c9640 1 -- 10.20.193.121:0/2753841909 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ff968004420 con 0x7ff980105b40 2026-02-13T10:39:10.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff9848c9640 1 -- 10.20.193.121:0/2753841909 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7ff968038c00 con 0x7ff980105b40 2026-02-13T10:39:10.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9801a1ac0 con 0x7ff980105b40 2026-02-13T10:39:10.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff9848c9640 1 -- 10.20.193.121:0/2753841909 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ff968041b10 con 0x7ff980105b40 2026-02-13T10:39:10.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9801a20d0 con 0x7ff980105b40 2026-02-13T10:39:10.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.065+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff980104dc0 con 0x7ff980105b40 2026-02-13T10:39:10.070 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.066+0000 7ff9848c9640 1 -- 10.20.193.121:0/2753841909 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7ff968038770 con 0x7ff980105b40 2026-02-13T10:39:10.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.066+0000 7ff9848c9640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9540775a0 0x7ff954079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:10.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.066+0000 7ff97effd640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9540775a0 0x7ff954079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:10.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.066+0000 7ff9848c9640 1 -- 10.20.193.121:0/2753841909 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4312+0+0 (secure 0 0 0) 0x7ff96803e070 con 0x7ff980105b40 2026-02-13T10:39:10.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.067+0000 7ff97effd640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9540775a0 0x7ff954079a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9801059a0 tx=0x7ff974008040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:10.071 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.068+0000 7ff9848c9640 1 -- 10.20.193.121:0/2753841909 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff968049340 con 0x7ff980105b40 2026-02-13T10:39:10.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.150+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff9801a24c0 con 0x7ff980105b40 2026-02-13T10:39:10.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.150+0000 7ff9848c9640 1 -- 10.20.193.121:0/2753841909 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v33) ==== 74+0+130 (secure 0 0 0) 0x7ff96808bbe0 con 0x7ff980105b40 2026-02-13T10:39:10.152 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:10.153 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9540775a0 msgr2=0x7ff954079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9540775a0 0x7ff954079a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9801059a0 tx=0x7ff974008040 comp rx=0 tx=0).stop 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 msgr2=0x7ff98019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff98019ca60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff968009ae0 tx=0x7ff968002e20 comp rx=0 tx=0).stop 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 shutdown_connections 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff9540775a0 0x7ff954079a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff980105b40 0x7ff98019ca60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 --2- 10.20.193.121:0/2753841909 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff980104940 0x7ff98019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 >> 10.20.193.121:0/2753841909 conn(0x7ff9801000f0 msgr2=0x7ff980108d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.151+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 shutdown_connections 2026-02-13T10:39:10.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:10.152+0000 7ff9858cb640 1 -- 10.20.193.121:0/2753841909 wait complete. 2026-02-13T10:39:10.190 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":33,"num_osds":6,"num_up_osds":5,"osd_up_since":1770979144,"num_in_osds":6,"osd_in_since":1770979139,"num_remapped_pgs":0} 2026-02-13T10:39:10.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: Detected new or changed devices on trial163 2026-02-13T10:39:10.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:10.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:39:10.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:39:10.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2026-02-13T10:39:10.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: Adjusting osd_memory_target on trial163 to 27290M 2026-02-13T10:39:10.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:10.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:10.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:10.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:09 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:11.191 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T10:39:11.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:10 trial121 ceph-mon[19159]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:11.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:10 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2753841909' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:11.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:10 trial163 ceph-mon[24184]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:11.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:10 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2753841909' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:11.479 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:11.589 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 -- 10.20.193.121:0/2776706150 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 msgr2=0x7fa33c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:11.589 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 --2- 10.20.193.121:0/2776706150 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c105fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3240099b0 tx=0x7fa32402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:11.589 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 -- 10.20.193.121:0/2776706150 shutdown_connections 2026-02-13T10:39:11.589 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 --2- 10.20.193.121:0/2776706150 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c105fc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:11.589 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 --2- 10.20.193.121:0/2776706150 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa33c104940 0x7fa33c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 -- 10.20.193.121:0/2776706150 >> 10.20.193.121:0/2776706150 conn(0x7fa33c1000f0 msgr2=0x7fa33c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 -- 10.20.193.121:0/2776706150 shutdown_connections 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.585+0000 7fa34104a640 1 -- 10.20.193.121:0/2776706150 wait complete. 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa34104a640 1 Processor -- start 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa34104a640 1 -- start start 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa34104a640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa33c104940 0x7fa33c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa34104a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa34104a640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10d610 con 0x7fa33c105b40 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa34104a640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10d490 con 0x7fa33c104940 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33a575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33a575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:35688/0 (socket says 10.20.193.121:35688) 2026-02-13T10:39:11.590 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33a575640 1 -- 10.20.193.121:0/3436982337 learned_addr learned my addr 10.20.193.121:0/3436982337 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33ad76640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa33c104940 0x7fa33c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33a575640 1 -- 10.20.193.121:0/3436982337 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa33c104940 msgr2=0x7fa33c19c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33a575640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa33c104940 0x7fa33c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33a575640 1 -- 10.20.193.121:0/3436982337 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c19cf90 con 0x7fa33c105b40 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa33a575640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c19c910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa32402fd00 tx=0x7fa32402fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa31bfff640 1 -- 10.20.193.121:0/3436982337 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fa324004420 con 0x7fa33c105b40 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa31bfff640 1 -- 10.20.193.121:0/3436982337 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fa32402fe50 con 0x7fa33c105b40 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c1a1970 con 0x7fa33c105b40 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.586+0000 7fa31bfff640 1 -- 10.20.193.121:0/3436982337 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fa324041a00 con 0x7fa33c105b40 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.587+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c1a1f80 con 0x7fa33c105b40 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.587+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c104d40 con 0x7fa33c105b40 2026-02-13T10:39:11.591 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.588+0000 7fa31bfff640 1 -- 10.20.193.121:0/3436982337 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fa324041ba0 con 0x7fa33c105b40 2026-02-13T10:39:11.592 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.588+0000 7fa31bfff640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa3100775a0 0x7fa310079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:11.593 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.588+0000 7fa33ad76640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa3100775a0 0x7fa310079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:11.593 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.588+0000 7fa31bfff640 1 -- 10.20.193.121:0/3436982337 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4312+0+0 (secure 0 0 0) 0x7fa32403e070 con 0x7fa33c105b40 2026-02-13T10:39:11.593 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.588+0000 7fa33ad76640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa3100775a0 0x7fa310079a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa330005fd0 tx=0x7fa330005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:11.593 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.590+0000 7fa31bfff640 1 -- 10.20.193.121:0/3436982337 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa324049340 con 0x7fa33c105b40 2026-02-13T10:39:11.673 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.672+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa33c196020 con 0x7fa33c105b40 2026-02-13T10:39:11.674 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.672+0000 7fa31bfff640 1 -- 10.20.193.121:0/3436982337 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v33) ==== 74+0+130 (secure 0 0 0) 0x7fa32408b950 con 0x7fa33c105b40 2026-02-13T10:39:11.674 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:11.675 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa3100775a0 msgr2=0x7fa310079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:11.675 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa3100775a0 0x7fa310079a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa330005fd0 tx=0x7fa330005950 comp rx=0 tx=0).stop 2026-02-13T10:39:11.675 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 msgr2=0x7fa33c19c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:11.675 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c19c910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa32402fd00 tx=0x7fa32402fd30 comp rx=0 tx=0).stop 2026-02-13T10:39:11.675 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 shutdown_connections 2026-02-13T10:39:11.676 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa3100775a0 0x7fa310079a60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:11.676 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa33c105b40 0x7fa33c19c910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:11.676 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 --2- 10.20.193.121:0/3436982337 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa33c104940 0x7fa33c19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:11.676 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 >> 10.20.193.121:0/3436982337 conn(0x7fa33c1000f0 msgr2=0x7fa33c109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:11.676 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 shutdown_connections 2026-02-13T10:39:11.676 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:11.673+0000 7fa34104a640 1 -- 10.20.193.121:0/3436982337 wait complete. 2026-02-13T10:39:11.711 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":33,"num_osds":6,"num_up_osds":5,"osd_up_since":1770979144,"num_in_osds":6,"osd_in_since":1770979139,"num_remapped_pgs":0} 2026-02-13T10:39:12.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:11 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3436982337' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:12.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:11 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3436982337' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:12.712 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T10:39:13.001 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:13.024 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:12 trial121 ceph-mon[19159]: pgmap v64: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:13.107 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/2549452026 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 msgr2=0x7f6a58105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/2549452026 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a58105fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0099b0 tx=0x7f6a4c02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/2549452026 shutdown_connections 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/2549452026 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a58105fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/2549452026 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6a58104940 0x7f6a58104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/2549452026 >> 10.20.193.121:0/2549452026 conn(0x7f6a581000f0 msgr2=0x7f6a58102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/2549452026 shutdown_connections 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.104+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/2549452026 wait complete. 2026-02-13T10:39:13.108 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 Processor -- start 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 -- start start 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6a58104940 0x7f6a5819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a5819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810d6b0 con 0x7f6a58105b40 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810d530 con 0x7f6a58104940 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5d265640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a5819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5d265640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a5819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:35714/0 (socket says 10.20.193.121:35714) 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5d265640 1 -- 10.20.193.121:0/557689194 learned_addr learned my addr 10.20.193.121:0/557689194 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5da66640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6a58104940 0x7f6a5819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5d265640 1 -- 10.20.193.121:0/557689194 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6a58104940 msgr2=0x7f6a5819c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:13.109 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5d265640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6a58104940 0x7f6a5819c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5d265640 1 -- 10.20.193.121:0/557689194 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5819d0c0 con 0x7f6a58105b40 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5d265640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a5819ca40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c038660 tx=0x7f6a4c002f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a46ffd640 1 -- 10.20.193.121:0/557689194 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6a4c046070 con 0x7f6a58105b40 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a46ffd640 1 -- 10.20.193.121:0/557689194 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f6a4c0044b0 con 0x7f6a58105b40 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a581a1aa0 con 0x7f6a58105b40 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a46ffd640 1 -- 10.20.193.121:0/557689194 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6a4c041a80 con 0x7f6a58105b40 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.105+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a581a20b0 con 0x7f6a58105b40 2026-02-13T10:39:13.110 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.106+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a58104dc0 con 0x7f6a58105b40 2026-02-13T10:39:13.111 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.106+0000 7f6a46ffd640 1 -- 10.20.193.121:0/557689194 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f6a4c038770 con 0x7f6a58105b40 2026-02-13T10:39:13.111 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.107+0000 7f6a46ffd640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6a34077550 0x7f6a34079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:13.111 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.107+0000 7f6a5da66640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6a34077550 0x7f6a34079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:13.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.107+0000 7f6a46ffd640 1 -- 10.20.193.121:0/557689194 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4312+0+0 (secure 0 0 0) 0x7f6a4c03e070 con 0x7f6a58105b40 2026-02-13T10:39:13.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.107+0000 7f6a5da66640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6a34077550 0x7f6a34079a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a581059a0 tx=0x7f6a48008040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:13.112 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.109+0000 7f6a46ffd640 1 -- 10.20.193.121:0/557689194 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6a4c08b710 con 0x7f6a58105b40 2026-02-13T10:39:13.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.194+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6a58063880 con 0x7f6a58105b40 2026-02-13T10:39:13.196 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.194+0000 7f6a46ffd640 1 -- 10.20.193.121:0/557689194 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v33) ==== 74+0+130 (secure 0 0 0) 0x7f6a4c0905c0 con 0x7f6a58105b40 2026-02-13T10:39:13.196 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:13.197 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6a34077550 msgr2=0x7f6a34079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:13.197 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6a34077550 0x7f6a34079a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a581059a0 tx=0x7f6a48008040 comp rx=0 tx=0).stop 2026-02-13T10:39:13.197 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 msgr2=0x7f6a5819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:13.197 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a5819ca40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c038660 tx=0x7f6a4c002f60 comp rx=0 tx=0).stop 2026-02-13T10:39:13.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 shutdown_connections 2026-02-13T10:39:13.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6a34077550 0x7f6a34079a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:13.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6a58105b40 0x7f6a5819ca40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:13.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 --2- 10.20.193.121:0/557689194 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6a58104940 0x7f6a5819c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:13.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 >> 10.20.193.121:0/557689194 conn(0x7f6a581000f0 msgr2=0x7f6a581091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:13.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 shutdown_connections 2026-02-13T10:39:13.198 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:13.195+0000 7f6a5fcf1640 1 -- 10.20.193.121:0/557689194 wait complete. 2026-02-13T10:39:13.231 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":33,"num_osds":6,"num_up_osds":5,"osd_up_since":1770979144,"num_in_osds":6,"osd_in_since":1770979139,"num_remapped_pgs":0} 2026-02-13T10:39:13.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:12 trial163 ceph-mon[24184]: pgmap v64: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:13.856 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:39:13 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:39:13.553+0000 7f0239ea0740 -1 osd.5 0 log_to_monitors true 2026-02-13T10:39:14.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:13 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/557689194' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:14.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:13 trial121 ceph-mon[19159]: from='osd.5 [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2026-02-13T10:39:14.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:13 trial121 ceph-mon[19159]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2026-02-13T10:39:14.233 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T10:39:14.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:13 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/557689194' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:14.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:13 trial163 ceph-mon[24184]: from='osd.5 [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2026-02-13T10:39:14.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:13 trial163 ceph-mon[24184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2026-02-13T10:39:14.522 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:14.628 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/3216495318 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec105b40 msgr2=0x7fa2ec105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/3216495318 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec105b40 0x7fa2ec105fc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40099b0 tx=0x7fa2d402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/3216495318 shutdown_connections 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/3216495318 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec105b40 0x7fa2ec105fc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/3216495318 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa2ec104940 0x7fa2ec104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/3216495318 >> 10.20.193.121:0/3216495318 conn(0x7fa2ec1000f0 msgr2=0x7fa2ec102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/3216495318 shutdown_connections 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/3216495318 wait complete. 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 Processor -- start 2026-02-13T10:39:14.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- start start 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec104940 0x7fa2ec19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa2ec105b40 0x7fa2ec19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec10d610 con 0x7fa2ec104940 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2f0ea2640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec10d490 con 0x7fa2ec105b40 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2ea575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec104940 0x7fa2ec19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2ea575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec104940 0x7fa2ec19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:35722/0 (socket says 10.20.193.121:35722) 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.625+0000 7fa2ea575640 1 -- 10.20.193.121:0/842596054 learned_addr learned my addr 10.20.193.121:0/842596054 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2e9d74640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa2ec105b40 0x7fa2ec19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2ea575640 1 -- 10.20.193.121:0/842596054 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa2ec105b40 msgr2=0x7fa2ec19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2ea575640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa2ec105b40 0x7fa2ec19c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2ea575640 1 -- 10.20.193.121:0/842596054 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec19cfc0 con 0x7fa2ec104940 2026-02-13T10:39:14.630 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2ea575640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec104940 0x7fa2ec19c400 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000d900 tx=0x7fa2e000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:14.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2d37fe640 1 -- 10.20.193.121:0/842596054 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fa2e0014070 con 0x7fa2ec104940 2026-02-13T10:39:14.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2d37fe640 1 -- 10.20.193.121:0/842596054 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fa2e00044a0 con 0x7fa2ec104940 2026-02-13T10:39:14.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec1a1a00 con 0x7fa2ec104940 2026-02-13T10:39:14.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2d37fe640 1 -- 10.20.193.121:0/842596054 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fa2e0002d40 con 0x7fa2ec104940 2026-02-13T10:39:14.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec1a2010 con 0x7fa2ec104940 2026-02-13T10:39:14.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.626+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ec104d40 con 0x7fa2ec104940 2026-02-13T10:39:14.631 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.627+0000 7fa2d37fe640 1 -- 10.20.193.121:0/842596054 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fa2e000b840 con 0x7fa2ec104940 2026-02-13T10:39:14.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.627+0000 7fa2d37fe640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa2c40775a0 0x7fa2c4079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:14.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.627+0000 7fa2d37fe640 1 -- 10.20.193.121:0/842596054 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4333+0+0 (secure 0 0 0) 0x7fa2e00999a0 con 0x7fa2ec104940 2026-02-13T10:39:14.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.627+0000 7fa2e9d74640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa2c40775a0 0x7fa2c4079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:14.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.628+0000 7fa2e9d74640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa2c40775a0 0x7fa2c4079a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40097c0 tx=0x7fa2d40023d0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:14.632 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.629+0000 7fa2d37fe640 1 -- 10.20.193.121:0/842596054 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa2e0067000 con 0x7fa2ec104940 2026-02-13T10:39:14.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.713+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa2ec195dd0 con 0x7fa2ec104940 2026-02-13T10:39:14.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.713+0000 7fa2d37fe640 1 -- 10.20.193.121:0/842596054 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v34) ==== 74+0+130 (secure 0 0 0) 0x7fa2e006beb0 con 0x7fa2ec104940 2026-02-13T10:39:14.715 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:14.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa2c40775a0 msgr2=0x7fa2c4079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:14.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa2c40775a0 0x7fa2c4079a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40097c0 tx=0x7fa2d40023d0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec104940 msgr2=0x7fa2ec19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:14.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec104940 0x7fa2ec19c400 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000d900 tx=0x7fa2e000ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 shutdown_connections 2026-02-13T10:39:14.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fa2c40775a0 0x7fa2c4079a60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fa2ec105b40 0x7fa2ec19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 --2- 10.20.193.121:0/842596054 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fa2ec104940 0x7fa2ec19c400 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:14.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 >> 10.20.193.121:0/842596054 conn(0x7fa2ec1000f0 msgr2=0x7fa2ec109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:14.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 shutdown_connections 2026-02-13T10:39:14.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:14.714+0000 7fa2f0ea2640 1 -- 10.20.193.121:0/842596054 wait complete. 2026-02-13T10:39:14.751 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":34,"num_osds":6,"num_up_osds":5,"osd_up_since":1770979144,"num_in_osds":6,"osd_in_since":1770979139,"num_remapped_pgs":0} 2026-02-13T10:39:15.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:14 trial121 ceph-mon[19159]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:15.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:14 trial121 ceph-mon[19159]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-13T10:39:15.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:14 trial121 ceph-mon[19159]: osdmap e34: 6 total, 5 up, 6 in 2026-02-13T10:39:15.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:14 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:15.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:14 trial121 ceph-mon[19159]: from='osd.5 [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:15.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:14 trial121 ceph-mon[19159]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:15.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:14 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/842596054' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:15.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:14 trial163 ceph-mon[24184]: pgmap v65: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:15.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:14 trial163 ceph-mon[24184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-13T10:39:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:14 trial163 ceph-mon[24184]: osdmap e34: 6 total, 5 up, 6 in 2026-02-13T10:39:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:14 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:14 trial163 ceph-mon[24184]: from='osd.5 [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:14 trial163 ceph-mon[24184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial163", "root=default"]}]: dispatch 2026-02-13T10:39:15.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:14 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/842596054' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:15.291 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:39:14 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:39:14.858+0000 7f0236634640 -1 osd.5 0 waiting for initial osdmap 2026-02-13T10:39:15.291 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:39:14 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:39:14.864+0000 7f022dc31640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T10:39:15.752 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T10:39:16.040 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:16.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 -- 10.20.193.121:0/895276023 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0078cf0 msgr2=0x7f5ab0079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 --2- 10.20.193.121:0/895276023 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0078cf0 0x7f5ab0079150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40099b0 tx=0x7f5aa402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:16.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 -- 10.20.193.121:0/895276023 shutdown_connections 2026-02-13T10:39:16.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 --2- 10.20.193.121:0/895276023 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0078cf0 0x7f5ab0079150 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 --2- 10.20.193.121:0/895276023 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5ab0077aa0 0x7f5ab0077ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 -- 10.20.193.121:0/895276023 >> 10.20.193.121:0/895276023 conn(0x7f5ab01000f0 msgr2=0x7f5ab0102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:16.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 -- 10.20.193.121:0/895276023 shutdown_connections 2026-02-13T10:39:16.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.144+0000 7f5ab791b640 1 -- 10.20.193.121:0/895276023 wait complete. 2026-02-13T10:39:16.147 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:15 trial121 ceph-mon[19159]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial163", "root=default"]}]': finished 2026-02-13T10:39:16.147 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:15 trial121 ceph-mon[19159]: osdmap e35: 6 total, 5 up, 6 in 2026-02-13T10:39:16.147 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:15 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:16.148 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:15 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:16.148 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:15 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:39:16.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 Processor -- start 2026-02-13T10:39:16.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 -- start start 2026-02-13T10:39:16.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0077aa0 0x7f5ab01a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:16.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5ab0078cf0 0x7f5ab01a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:16.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0111b00 con 0x7f5ab0077aa0 2026-02-13T10:39:16.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0111980 con 0x7f5ab0078cf0 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab5690640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0077aa0 0x7f5ab01a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab5690640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0077aa0 0x7f5ab01a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:35744/0 (socket says 10.20.193.121:35744) 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab5690640 1 -- 10.20.193.121:0/1785047436 learned_addr learned my addr 10.20.193.121:0/1785047436 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab4e8f640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5ab0078cf0 0x7f5ab01a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab5690640 1 -- 10.20.193.121:0/1785047436 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5ab0078cf0 msgr2=0x7f5ab01a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab5690640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5ab0078cf0 0x7f5ab01a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab5690640 1 -- 10.20.193.121:0/1785047436 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab01a1480 con 0x7f5ab0077aa0 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab4e8f640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5ab0078cf0 0x7f5ab01a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab5690640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0077aa0 0x7f5ab01a08c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000d8d0 tx=0x7f5aa000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5a9e7fc640 1 -- 10.20.193.121:0/1785047436 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5aa0014070 con 0x7f5ab0077aa0 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5a9e7fc640 1 -- 10.20.193.121:0/1785047436 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f5aa00044a0 con 0x7f5ab0077aa0 2026-02-13T10:39:16.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5a9e7fc640 1 -- 10.20.193.121:0/1785047436 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5aa0002e60 con 0x7f5ab0077aa0 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab01a5ec0 con 0x7f5ab0077aa0 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.145+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab01a63f0 con 0x7f5ab0077aa0 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.146+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab0077ea0 con 0x7f5ab0077aa0 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.147+0000 7f5a9e7fc640 1 -- 10.20.193.121:0/1785047436 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f5aa000b840 con 0x7f5ab0077aa0 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.147+0000 7f5a9e7fc640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a8c0775a0 0x7f5a8c079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.147+0000 7f5a9e7fc640 1 -- 10.20.193.121:0/1785047436 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4629+0+0 (secure 0 0 0) 0x7f5aa0099690 con 0x7f5ab0077aa0 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.147+0000 7f5ab4e8f640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a8c0775a0 0x7f5a8c079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:16.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.147+0000 7f5ab4e8f640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a8c0775a0 0x7f5a8c079a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40097c0 tx=0x7f5aa4005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:16.153 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.149+0000 7f5a9e7fc640 1 -- 10.20.193.121:0/1785047436 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5aa0066b00 con 0x7f5ab0077aa0 2026-02-13T10:39:16.233 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.231+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5ab019a570 con 0x7f5ab0077aa0 2026-02-13T10:39:16.233 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5a9e7fc640 1 -- 10.20.193.121:0/1785047436 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v36) ==== 74+0+130 (secure 0 0 0) 0x7f5aa006b9b0 con 0x7f5ab0077aa0 2026-02-13T10:39:16.233 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:16.234 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a8c0775a0 msgr2=0x7f5a8c079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a8c0775a0 0x7f5a8c079a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40097c0 tx=0x7f5aa4005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0077aa0 msgr2=0x7f5ab01a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0077aa0 0x7f5ab01a08c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000d8d0 tx=0x7f5aa000dda0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 shutdown_connections 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a8c0775a0 0x7f5a8c079a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5ab0078cf0 0x7f5ab01a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 --2- 10.20.193.121:0/1785047436 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5ab0077aa0 0x7f5ab01a08c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 >> 10.20.193.121:0/1785047436 conn(0x7f5ab01000f0 msgr2=0x7f5ab010f170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 shutdown_connections 2026-02-13T10:39:16.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.232+0000 7f5ab791b640 1 -- 10.20.193.121:0/1785047436 wait complete. 2026-02-13T10:39:16.270 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":36,"num_osds":6,"num_up_osds":6,"osd_up_since":1770979155,"num_in_osds":6,"osd_in_since":1770979139,"num_remapped_pgs":0} 2026-02-13T10:39:16.271 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T10:39:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:15 trial163 ceph-mon[24184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial163", "root=default"]}]': finished 2026-02-13T10:39:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:15 trial163 ceph-mon[24184]: osdmap e35: 6 total, 5 up, 6 in 2026-02-13T10:39:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:15 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:16.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:15 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:16.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:15 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:39:16.558 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:16.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- 10.20.193.121:0/512728320 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 msgr2=0x7f8e44104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/512728320 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e44104d20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8e380099b0 tx=0x7f8e3802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:16.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- 10.20.193.121:0/512728320 shutdown_connections 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/512728320 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e44104d20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/512728320 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8e4410a8c0 0x7f8e4410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- 10.20.193.121:0/512728320 >> 10.20.193.121:0/512728320 conn(0x7f8e441005c0 msgr2=0x7f8e441029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- 10.20.193.121:0/512728320 shutdown_connections 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- 10.20.193.121:0/512728320 wait complete. 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 Processor -- start 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- start start 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e441a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8e4410a8c0 0x7f8e441a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44111c40 con 0x7f8e441048c0 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44111ac0 con 0x7f8e4410a8c0 2026-02-13T10:39:16.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e497ea640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e441a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e497ea640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e441a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:35762/0 (socket says 10.20.193.121:35762) 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e497ea640 1 -- 10.20.193.121:0/1833185660 learned_addr learned my addr 10.20.193.121:0/1833185660 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e497ea640 1 -- 10.20.193.121:0/1833185660 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8e4410a8c0 msgr2=0x7f8e441a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e497ea640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8e4410a8c0 0x7f8e441a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e497ea640 1 -- 10.20.193.121:0/1833185660 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4419c750 con 0x7f8e441048c0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e497ea640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e441a2680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e9b0 tx=0x7f8e3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e327fc640 1 -- 10.20.193.121:0/1833185660 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f8e3400cbe0 con 0x7f8e441048c0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e327fc640 1 -- 10.20.193.121:0/1833185660 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f8e34004560 con 0x7f8e441048c0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4419ca40 con 0x7f8e441048c0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e327fc640 1 -- 10.20.193.121:0/1833185660 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f8e34010430 con 0x7f8e441048c0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.661+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4419cfa0 con 0x7f8e441048c0 2026-02-13T10:39:16.666 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.662+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e44106000 con 0x7f8e441048c0 2026-02-13T10:39:16.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.663+0000 7f8e327fc640 1 -- 10.20.193.121:0/1833185660 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f8e340040d0 con 0x7f8e441048c0 2026-02-13T10:39:16.667 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.663+0000 7f8e327fc640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8e200775a0 0x7f8e20079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:16.668 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.663+0000 7f8e48fe9640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8e200775a0 0x7f8e20079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:16.668 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.663+0000 7f8e327fc640 1 -- 10.20.193.121:0/1833185660 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4629+0+0 (secure 0 0 0) 0x7f8e340997d0 con 0x7f8e441048c0 2026-02-13T10:39:16.668 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.663+0000 7f8e48fe9640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8e200775a0 0x7f8e20079a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e38005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:16.668 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.665+0000 7f8e327fc640 1 -- 10.20.193.121:0/1833185660 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8e34066d00 con 0x7f8e441048c0 2026-02-13T10:39:16.750 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.747+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e44104d20 con 0x7f8e441048c0 2026-02-13T10:39:16.750 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.748+0000 7f8e327fc640 1 -- 10.20.193.121:0/1833185660 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+11428 (secure 0 0 0) 0x7f8e3406bbb0 con 0x7f8e441048c0 2026-02-13T10:39:16.750 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:16.750 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":36,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","created":"2026-02-13T10:37:09.311059+0000","modified":"2026-02-13T10:39:15.860548+0000","last_up_change":"2026-02-13T10:39:15.860548+0000","last_in_change":"2026-02-13T10:38:59.652055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T10:38:43.586630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15b6509f-b28f-4bda-bbdc-c0c0b3074fca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6802","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6803","nonce":1486620450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6804","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6805","nonce":1486620450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6808","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6809","nonce":1486620450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6806","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6807","nonce":1486620450}]},"public_addr":"10.20.193.121:6803/1486620450","cluster_addr":"10.20.193.121:6805/1486620450","heartbeat_back_addr":"10.20.193.121:6809/1486620450","heartbeat_front_addr":"10.20.193.121:6807/1486620450","state":["exists","up"]},{"osd":1,"uuid":"c3e08c6d-c2cb-4d19-988c-7ec1d5d70aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6810","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6811","nonce":4000550628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6812","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6813","nonce":4000550628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6816","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6817","nonce":4000550628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6814","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6815","nonce":4000550628}]},"public_addr":"10.20.193.121:6811/4000550628","cluster_addr":"10.20.193.121:6813/4000550628","heartbeat_back_addr":"10.20.193.121:6817/4000550628","heartbeat_front_addr":"10.20.193.121:6815/4000550628","state":["exists","up"]},{"osd":2,"uuid":"3aad7001-dd1d-4ab3-bee7-17801f2b1563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6818","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6819","nonce":3599178802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6820","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6821","nonce":3599178802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6824","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6825","nonce":3599178802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6822","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6823","nonce":3599178802}]},"public_addr":"10.20.193.121:6819/3599178802","cluster_addr":"10.20.193.121:6821/3599178802","heartbeat_back_addr":"10.20.193.121:6825/3599178802","heartbeat_front_addr":"10.20.193.121:6823/3599178802","state":["exists","up"]},{"osd":3,"uuid":"09d5bd71-44e9-4c3f-a3b6-f15074495bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6800","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6801","nonce":2365834735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6802","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6803","nonce":2365834735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6806","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6807","nonce":2365834735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6804","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6805","nonce":2365834735}]},"public_addr":"10.20.193.163:6801/2365834735","cluster_addr":"10.20.193.163:6803/2365834735","heartbeat_back_addr":"10.20.193.163:6807/2365834735","heartbeat_front_addr":"10.20.193.163:6805/2365834735","state":["exists","up"]},{"osd":4,"uuid":"7d965c3c-65dd-4fd9-a1ee-996241c8d2e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6808","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6809","nonce":3393667471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6810","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6811","nonce":3393667471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6814","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6815","nonce":3393667471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6812","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6813","nonce":3393667471}]},"public_addr":"10.20.193.163:6809/3393667471","cluster_addr":"10.20.193.163:6811/3393667471","heartbeat_back_addr":"10.20.193.163:6815/3393667471","heartbeat_front_addr":"10.20.193.163:6813/3393667471","state":["exists","up"]},{"osd":5,"uuid":"d5e8a5b3-c3d6-4c49-97ce-8641644d24cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6816","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6817","nonce":2139375176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6818","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6819","nonce":2139375176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6822","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6823","nonce":2139375176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6820","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6821","nonce":2139375176}]},"public_addr":"10.20.193.163:6817/2139375176","cluster_addr":"10.20.193.163:6819/2139375176","heartbeat_back_addr":"10.20.193.163:6823/2139375176","heartbeat_front_addr":"10.20.193.163:6821/2139375176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:19.148789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:30.586490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:41.413339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:52.330028+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:39:03.516006+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.121:0/1986687891":"2026-02-14T10:37:45.544561+0000","10.20.193.121:0/243024684":"2026-02-14T10:37:19.150822+0000","10.20.193.121:6801/232220030":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6801/3394068414":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/1469338567":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/1306451008":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/1699876434":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/3836430930":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/1389694817":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/2646306236":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6801/977129689":"2026-02-14T10:37:28.590631+0000","10.20.193.121:6800/977129689":"2026-02-14T10:37:28.590631+0000","10.20.193.121:6800/3394068414":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/3763370872":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6800/232220030":"2026-02-14T10:37:45.544561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T10:39:16.751 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8e200775a0 msgr2=0x7f8e20079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8e200775a0 0x7f8e20079a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e38005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 msgr2=0x7f8e441a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e441a2680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e9b0 tx=0x7f8e3400ee80 comp rx=0 tx=0).stop 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 shutdown_connections 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8e200775a0 0x7f8e20079a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8e4410a8c0 0x7f8e441a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 --2- 10.20.193.121:0/1833185660 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8e441048c0 0x7f8e441a2680 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 >> 10.20.193.121:0/1833185660 conn(0x7f8e441005c0 msgr2=0x7f8e441016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 shutdown_connections 2026-02-13T10:39:16.752 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:16.749+0000 7f8e4ba75640 1 -- 10.20.193.121:0/1833185660 wait complete. 2026-02-13T10:39:16.789 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-13T10:38:43.586630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-13T10:39:16.789 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd pool get .mgr pg_num 2026-02-13T10:39:17.078 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:17.102 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: purged_snaps scrub starts 2026-02-13T10:39:17.102 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: purged_snaps scrub ok 2026-02-13T10:39:17.103 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:17.103 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:17.103 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: osd.5 [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] boot 2026-02-13T10:39:17.103 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: osdmap e36: 6 total, 6 up, 6 in 2026-02-13T10:39:17.103 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:17.103 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1785047436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:17.103 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:16 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1833185660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2026-02-13T10:39:17.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- 10.20.193.121:0/704025783 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077620 msgr2=0x7fcff8077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 --2- 10.20.193.121:0/704025783 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077620 0x7fcff8077a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80099b0 tx=0x7fcfe802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:17.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- 10.20.193.121:0/704025783 shutdown_connections 2026-02-13T10:39:17.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 --2- 10.20.193.121:0/704025783 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcff8077f40 0x7fcff8113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 --2- 10.20.193.121:0/704025783 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077620 0x7fcff8077a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- 10.20.193.121:0/704025783 >> 10.20.193.121:0/704025783 conn(0x7fcff81005f0 msgr2=0x7fcff8102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- 10.20.193.121:0/704025783 shutdown_connections 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- 10.20.193.121:0/704025783 wait complete. 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 Processor -- start 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- start start 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcff8077620 0x7fcff81a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077f40 0x7fcff81a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8116170 con 0x7fcff8077f40 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fd0000fa640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8115ff0 con 0x7fcff8077620 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fcffd66e640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077f40 0x7fcff81a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:17.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fcffd66e640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077f40 0x7fcff81a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50286/0 (socket says 10.20.193.121:50286) 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.183+0000 7fcffd66e640 1 -- 10.20.193.121:0/2187965941 learned_addr learned my addr 10.20.193.121:0/2187965941 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fcffd66e640 1 -- 10.20.193.121:0/2187965941 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcff8077620 msgr2=0x7fcff81a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fcffd66e640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcff8077620 0x7fcff81a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fcffd66e640 1 -- 10.20.193.121:0/2187965941 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff81a5740 con 0x7fcff8077f40 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fcffd66e640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077f40 0x7fcff81a13b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00d900 tx=0x7fcfec00ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fcfe6ffd640 1 -- 10.20.193.121:0/2187965941 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fcfec014070 con 0x7fcff8077f40 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81a5a30 con 0x7fcff8077f40 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fcfe6ffd640 1 -- 10.20.193.121:0/2187965941 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fcfec0044a0 con 0x7fcff8077f40 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fcfe6ffd640 1 -- 10.20.193.121:0/2187965941 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fcfec005020 con 0x7fcff8077f40 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81a5fc0 con 0x7fcff8077f40 2026-02-13T10:39:17.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.184+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff8078d60 con 0x7fcff8077f40 2026-02-13T10:39:17.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.185+0000 7fcfe6ffd640 1 -- 10.20.193.121:0/2187965941 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fcfec005290 con 0x7fcff8077f40 2026-02-13T10:39:17.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.185+0000 7fcfe6ffd640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcfd40775a0 0x7fcfd4079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:17.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.186+0000 7fcfe6ffd640 1 -- 10.20.193.121:0/2187965941 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7fcfec099770 con 0x7fcff8077f40 2026-02-13T10:39:17.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.186+0000 7fcffde6f640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcfd40775a0 0x7fcfd4079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:17.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.186+0000 7fcffde6f640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcfd40775a0 0x7fcfd4079a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80097c0 tx=0x7fcfe8005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:17.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.187+0000 7fcfe6ffd640 1 -- 10.20.193.121:0/2187965941 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcfec066be0 con 0x7fcff8077f40 2026-02-13T10:39:17.271 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.269+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcff8077a00 con 0x7fcff8077f40 2026-02-13T10:39:17.271 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.270+0000 7fcfe6ffd640 1 -- 10.20.193.121:0/2187965941 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7fcfec06ba90 con 0x7fcff8077f40 2026-02-13T10:39:17.272 INFO:teuthology.orchestra.run.trial121.stdout:pg_num: 1 2026-02-13T10:39:17.272 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.270+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcfd40775a0 msgr2=0x7fcfd4079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.270+0000 7fd0000fa640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcfd40775a0 0x7fcfd4079a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80097c0 tx=0x7fcfe8005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.270+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077f40 msgr2=0x7fcff81a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.270+0000 7fd0000fa640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077f40 0x7fcff81a13b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00d900 tx=0x7fcfec00ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.271+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 shutdown_connections 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.271+0000 7fd0000fa640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcfd40775a0 0x7fcfd4079a60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.271+0000 7fd0000fa640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcff8077f40 0x7fcff81a13b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.271+0000 7fd0000fa640 1 --2- 10.20.193.121:0/2187965941 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcff8077620 0x7fcff81a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.271+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 >> 10.20.193.121:0/2187965941 conn(0x7fcff81005f0 msgr2=0x7fcff8101330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.271+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 shutdown_connections 2026-02-13T10:39:17.273 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.271+0000 7fd0000fa640 1 -- 10.20.193.121:0/2187965941 wait complete. 2026-02-13T10:39:17.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: purged_snaps scrub starts 2026-02-13T10:39:17.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: purged_snaps scrub ok 2026-02-13T10:39:17.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T10:39:17.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:17.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: osd.5 [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] boot 2026-02-13T10:39:17.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: osdmap e36: 6 total, 6 up, 6 in 2026-02-13T10:39:17.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2026-02-13T10:39:17.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1785047436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2026-02-13T10:39:17.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:16 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1833185660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2026-02-13T10:39:17.310 INFO:tasks.cephadm:Setting up client nodes... 2026-02-13T10:39:17.310 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2026-02-13T10:39:17.597 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 -- 10.20.193.121:0/4202151600 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109e50 msgr2=0x7f38f010a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 --2- 10.20.193.121:0/4202151600 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109e50 0x7f38f010a2d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f38e00099b0 tx=0x7f38e002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 -- 10.20.193.121:0/4202151600 shutdown_connections 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 --2- 10.20.193.121:0/4202151600 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109e50 0x7f38f010a2d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 --2- 10.20.193.121:0/4202151600 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f38f0109480 0x7f38f0109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 -- 10.20.193.121:0/4202151600 >> 10.20.193.121:0/4202151600 conn(0x7f38f006bf00 msgr2=0x7f38f007a790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 -- 10.20.193.121:0/4202151600 shutdown_connections 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.710+0000 7f38f6081640 1 -- 10.20.193.121:0/4202151600 wait complete. 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 Processor -- start 2026-02-13T10:39:17.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 -- start start 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109480 0x7f38f01a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f38f0109e50 0x7f38f01a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f38f0111b10 con 0x7f38f0109480 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f38f0111990 con 0x7f38f0109e50 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ef7fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109480 0x7f38f01a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ef7fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109480 0x7f38f01a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50302/0 (socket says 10.20.193.121:50302) 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ef7fe640 1 -- 10.20.193.121:0/2928219417 learned_addr learned my addr 10.20.193.121:0/2928219417 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38eeffd640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f38f0109e50 0x7f38f01a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ef7fe640 1 -- 10.20.193.121:0/2928219417 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f38f0109e50 msgr2=0x7f38f01a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ef7fe640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f38f0109e50 0x7f38f01a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ef7fe640 1 -- 10.20.193.121:0/2928219417 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f01a14a0 con 0x7f38f0109480 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38eeffd640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f38f0109e50 0x7f38f01a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ef7fe640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109480 0x7f38f01a08e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d900 tx=0x7f38dc00ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ecff9640 1 -- 10.20.193.121:0/2928219417 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f38dc014070 con 0x7f38f0109480 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f01a5ee0 con 0x7f38f0109480 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ecff9640 1 -- 10.20.193.121:0/2928219417 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f38dc0044a0 con 0x7f38f0109480 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38ecff9640 1 -- 10.20.193.121:0/2928219417 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f38dc002d40 con 0x7f38f0109480 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.711+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f01a6410 con 0x7f38f0109480 2026-02-13T10:39:17.716 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.712+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f0109880 con 0x7f38f0109480 2026-02-13T10:39:17.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.712+0000 7f38ecff9640 1 -- 10.20.193.121:0/2928219417 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f38dc00b840 con 0x7f38f0109480 2026-02-13T10:39:17.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.713+0000 7f38ecff9640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f38c40775a0 0x7f38c4079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:17.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.713+0000 7f38ecff9640 1 -- 10.20.193.121:0/2928219417 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f38dc099b20 con 0x7f38f0109480 2026-02-13T10:39:17.717 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.713+0000 7f38eeffd640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f38c40775a0 0x7f38c4079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:17.718 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.713+0000 7f38eeffd640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f38c40775a0 0x7f38c4079a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f38e00097c0 tx=0x7f38e0005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:17.718 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.715+0000 7f38ecff9640 1 -- 10.20.193.121:0/2928219417 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f38dc066f90 con 0x7f38f0109480 2026-02-13T10:39:17.829 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.828+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f38f019a2d0 con 0x7f38f0109480 2026-02-13T10:39:17.830 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.829+0000 7f38ecff9640 1 -- 10.20.193.121:0/2928219417 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) ==== 170+0+59 (secure 0 0 0) 0x7f38dc06be40 con 0x7f38f0109480 2026-02-13T10:39:17.831 INFO:teuthology.orchestra.run.trial121.stdout:[client.0] 2026-02-13T10:39:17.831 INFO:teuthology.orchestra.run.trial121.stdout: key = AQBV/45p3kZsMRAAQH7rqRr9R5Z/4opRn/0HVA== 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f38c40775a0 msgr2=0x7f38c4079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f38c40775a0 0x7f38c4079a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f38e00097c0 tx=0x7f38e0005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109480 msgr2=0x7f38f01a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109480 0x7f38f01a08e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d900 tx=0x7f38dc00ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 shutdown_connections 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f38c40775a0 0x7f38c4079a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f38f0109e50 0x7f38f01a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 --2- 10.20.193.121:0/2928219417 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f38f0109480 0x7f38f01a08e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 >> 10.20.193.121:0/2928219417 conn(0x7f38f006bf00 msgr2=0x7f38f010d830 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 shutdown_connections 2026-02-13T10:39:17.832 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:17.830+0000 7f38f6081640 1 -- 10.20.193.121:0/2928219417 wait complete. 2026-02-13T10:39:17.887 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:17 trial121 ceph-mon[19159]: osdmap e37: 6 total, 6 up, 6 in 2026-02-13T10:39:17.887 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:17 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2187965941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2026-02-13T10:39:17.887 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:17 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2928219417' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2026-02-13T10:39:17.888 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:17 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2928219417' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-13T10:39:17.889 DEBUG:teuthology.orchestra.run.trial121:> set -ex 2026-02-13T10:39:17.889 DEBUG:teuthology.orchestra.run.trial121:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2026-02-13T10:39:17.889 DEBUG:teuthology.orchestra.run.trial121:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-13T10:39:17.925 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2026-02-13T10:39:18.214 INFO:teuthology.orchestra.run.trial163.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial163/config 2026-02-13T10:39:18.237 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:17 trial163 ceph-mon[24184]: osdmap e37: 6 total, 6 up, 6 in 2026-02-13T10:39:18.237 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:17 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2187965941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2026-02-13T10:39:18.237 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:17 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2928219417' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2026-02-13T10:39:18.237 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:17 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2928219417' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 -- 10.20.193.163:0/4278461892 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4077aa0 msgr2=0x7f0fd4077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 --2- 10.20.193.163:0/4278461892 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4077aa0 0x7f0fd4077ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40099b0 tx=0x7f0fc402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 -- 10.20.193.163:0/4278461892 shutdown_connections 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 --2- 10.20.193.163:0/4278461892 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0fd4078cf0 0x7f0fd4079150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 --2- 10.20.193.163:0/4278461892 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4077aa0 0x7f0fd4077ea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 -- 10.20.193.163:0/4278461892 >> 10.20.193.163:0/4278461892 conn(0x7f0fd41000d0 msgr2=0x7f0fd4102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 -- 10.20.193.163:0/4278461892 shutdown_connections 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.320+0000 7f0fdb354640 1 -- 10.20.193.163:0/4278461892 wait complete. 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 Processor -- start 2026-02-13T10:39:18.324 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 -- start start 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0fd4077aa0 0x7f0fd41a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4078cf0 0x7f0fd41a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4111b00 con 0x7f0fd4077aa0 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4111980 con 0x7f0fd4078cf0 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd88c8640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4078cf0 0x7f0fd41a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd88c8640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4078cf0 0x7f0fd41a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.163:3300/0 says I am v2:10.20.193.163:46156/0 (socket says 10.20.193.163:46156) 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd88c8640 1 -- 10.20.193.163:0/3398552114 learned_addr learned my addr 10.20.193.163:0/3398552114 (peer_addr_for_me v2:10.20.193.163:0/0) 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd90c9640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0fd4077aa0 0x7f0fd41a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd88c8640 1 -- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0fd4077aa0 msgr2=0x7f0fd41a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd88c8640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0fd4077aa0 0x7f0fd41a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd88c8640 1 -- 10.20.193.163:0/3398552114 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd41a14d0 con 0x7f0fd4078cf0 2026-02-13T10:39:18.325 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd90c9640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0fd4077aa0 0x7f0fd41a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:18.326 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fd88c8640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4078cf0 0x7f0fd41a0e50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800d8d0 tx=0x7f0fc800dda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:18.326 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fc27fc640 1 -- 10.20.193.163:0/3398552114 <== mon.1 v2:10.20.193.163:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f0fc8014070 con 0x7f0fd4078cf0 2026-02-13T10:39:18.326 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd41a5f10 con 0x7f0fd4078cf0 2026-02-13T10:39:18.326 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fc27fc640 1 -- 10.20.193.163:0/3398552114 <== mon.1 v2:10.20.193.163:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f0fc80044a0 con 0x7f0fd4078cf0 2026-02-13T10:39:18.326 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fc27fc640 1 -- 10.20.193.163:0/3398552114 <== mon.1 v2:10.20.193.163:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f0fc8002e60 con 0x7f0fd4078cf0 2026-02-13T10:39:18.326 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.321+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd41a6490 con 0x7f0fd4078cf0 2026-02-13T10:39:18.326 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.322+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd4077ea0 con 0x7f0fd4078cf0 2026-02-13T10:39:18.327 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.322+0000 7f0fc27fc640 1 -- 10.20.193.163:0/3398552114 <== mon.1 v2:10.20.193.163:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f0fc800b840 con 0x7f0fd4078cf0 2026-02-13T10:39:18.327 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.323+0000 7f0fc27fc640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0fb0077540 0x7f0fb0079a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:18.327 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.323+0000 7f0fc27fc640 1 -- 10.20.193.163:0/3398552114 <== mon.1 v2:10.20.193.163:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f0fc8099670 con 0x7f0fd4078cf0 2026-02-13T10:39:18.328 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.323+0000 7f0fd90c9640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0fb0077540 0x7f0fb0079a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:18.328 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.324+0000 7f0fd90c9640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0fb0077540 0x7f0fb0079a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40097c0 tx=0x7f0fc4005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:18.328 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.325+0000 7f0fc27fc640 1 -- 10.20.193.163:0/3398552114 <== mon.1 v2:10.20.193.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0fc8066b00 con 0x7f0fd4078cf0 2026-02-13T10:39:18.440 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.438+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0fd419a2c0 con 0x7f0fd4078cf0 2026-02-13T10:39:18.443 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.441+0000 7f0fc27fc640 1 -- 10.20.193.163:0/3398552114 <== mon.1 v2:10.20.193.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7f0fc806b9b0 con 0x7f0fd4078cf0 2026-02-13T10:39:18.443 INFO:teuthology.orchestra.run.trial163.stdout:[client.1] 2026-02-13T10:39:18.443 INFO:teuthology.orchestra.run.trial163.stdout: key = AQBW/45pB8o7GhAAlPTZh7HrUU0AxwPvcdSiYw== 2026-02-13T10:39:18.444 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0fb0077540 msgr2=0x7f0fb0079a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:18.444 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0fb0077540 0x7f0fb0079a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40097c0 tx=0x7f0fc4005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:18.444 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4078cf0 msgr2=0x7f0fd41a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:18.444 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4078cf0 0x7f0fd41a0e50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800d8d0 tx=0x7f0fc800dda0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.444 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 shutdown_connections 2026-02-13T10:39:18.444 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f0fb0077540 0x7f0fb0079a00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.445 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f0fd4078cf0 0x7f0fd41a0e50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.445 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 --2- 10.20.193.163:0/3398552114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f0fd4077aa0 0x7f0fd41a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.445 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 >> 10.20.193.163:0/3398552114 conn(0x7f0fd41000d0 msgr2=0x7f0fd410f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:18.445 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 shutdown_connections 2026-02-13T10:39:18.445 INFO:teuthology.orchestra.run.trial163.stderr:2026-02-13T10:39:18.442+0000 7f0fdb354640 1 -- 10.20.193.163:0/3398552114 wait complete. 2026-02-13T10:39:18.480 DEBUG:teuthology.orchestra.run.trial163:> set -ex 2026-02-13T10:39:18.480 DEBUG:teuthology.orchestra.run.trial163:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2026-02-13T10:39:18.480 DEBUG:teuthology.orchestra.run.trial163:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-13T10:39:18.515 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph config log 1 --format=json 2026-02-13T10:39:18.802 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:18.914 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- 10.20.193.121:0/40248989 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 msgr2=0x7f66a81079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 --2- 10.20.193.121:0/40248989 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a81079d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66980099b0 tx=0x7f669802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- 10.20.193.121:0/40248989 shutdown_connections 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 --2- 10.20.193.121:0/40248989 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f66a8101540 0x7f66a81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 --2- 10.20.193.121:0/40248989 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a81079d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- 10.20.193.121:0/40248989 >> 10.20.193.121:0/40248989 conn(0x7f66a80fd220 msgr2=0x7f66a80ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- 10.20.193.121:0/40248989 shutdown_connections 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- 10.20.193.121:0/40248989 wait complete. 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 Processor -- start 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- start start 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f66a8101540 0x7f66a8074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:18.915 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a806c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810e890 con 0x7f66a81075f0 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66b0361640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810e710 con 0x7f66a8101540 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66ad8d5640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66ad8d5640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a806c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50310/0 (socket says 10.20.193.121:50310) 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.911+0000 7f66ad8d5640 1 -- 10.20.193.121:0/3319502972 learned_addr learned my addr 10.20.193.121:0/3319502972 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66ae0d6640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f66a8101540 0x7f66a8074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66ad8d5640 1 -- 10.20.193.121:0/3319502972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f66a8101540 msgr2=0x7f66a8074990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66ad8d5640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f66a8101540 0x7f66a8074990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66ad8d5640 1 -- 10.20.193.121:0/3319502972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a8075010 con 0x7f66a81075f0 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66ad8d5640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a806c960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f669c00c910 tx=0x7f669c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:18.916 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f668f7fe640 1 -- 10.20.193.121:0/3319502972 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f669c007c30 con 0x7f66a81075f0 2026-02-13T10:39:18.917 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f668f7fe640 1 -- 10.20.193.121:0/3319502972 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f669c007dd0 con 0x7f66a81075f0 2026-02-13T10:39:18.917 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a806cf00 con 0x7f66a81075f0 2026-02-13T10:39:18.917 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f668f7fe640 1 -- 10.20.193.121:0/3319502972 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f669c00f450 con 0x7f66a81075f0 2026-02-13T10:39:18.917 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a806d490 con 0x7f66a81075f0 2026-02-13T10:39:18.917 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.912+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a806d7a0 con 0x7f66a81075f0 2026-02-13T10:39:18.917 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.913+0000 7f668f7fe640 1 -- 10.20.193.121:0/3319502972 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f669c005000 con 0x7f66a81075f0 2026-02-13T10:39:18.918 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.913+0000 7f668f7fe640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f66840775a0 0x7f6684079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:18.918 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.913+0000 7f668f7fe640 1 -- 10.20.193.121:0/3319502972 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f669c0998a0 con 0x7f66a81075f0 2026-02-13T10:39:18.918 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.913+0000 7f66ae0d6640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f66840775a0 0x7f6684079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:18.918 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.914+0000 7f66ae0d6640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f66840775a0 0x7f6684079a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66980097c0 tx=0x7f6698005990 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:18.919 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.915+0000 7f668f7fe640 1 -- 10.20.193.121:0/3319502972 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f669c014030 con 0x7f66a81075f0 2026-02-13T10:39:18.998 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.997+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f66a8102c60 con 0x7f66a81075f0 2026-02-13T10:39:18.999 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.997+0000 7f668f7fe640 1 -- 10.20.193.121:0/3319502972 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v39) ==== 86+0+188 (secure 0 0 0) 0x7f669c066dd0 con 0x7f66a81075f0 2026-02-13T10:39:18.999 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:19.000 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f66840775a0 msgr2=0x7f6684079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.000 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f66840775a0 0x7f6684079a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66980097c0 tx=0x7f6698005990 comp rx=0 tx=0).stop 2026-02-13T10:39:19.000 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 msgr2=0x7f66a806c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.000 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a806c960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f669c00c910 tx=0x7f669c00cde0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.000 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 shutdown_connections 2026-02-13T10:39:19.000 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f66840775a0 0x7f6684079a60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.000 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f66a81075f0 0x7f66a806c960 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.001 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 --2- 10.20.193.121:0/3319502972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f66a8101540 0x7f66a8074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.001 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 >> 10.20.193.121:0/3319502972 conn(0x7f66a80fd220 msgr2=0x7f66a806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:19.001 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 shutdown_connections 2026-02-13T10:39:19.001 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:18.998+0000 7f66b0361640 1 -- 10.20.193.121:0/3319502972 wait complete. 2026-02-13T10:39:19.037 INFO:teuthology.orchestra.run.trial121.stdout:[{"version":39,"timestamp":"2026-02-13T10:39:08.840797+0000","name":"","changes":[{"name":"osd/host:trial163/osd_memory_target","previous_value":"42924113715","new_value":"28616075810"}]}] 2026-02-13T10:39:19.038 INFO:tasks.ceph_manager:config epoch is 39 2026-02-13T10:39:19.038 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-13T10:39:19.038 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2026-02-13T10:39:19.038 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mgr dump --format=json 2026-02-13T10:39:19.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:18 trial121 ceph-mon[19159]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:19.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:18 trial121 ceph-mon[19159]: from='client.? 10.20.193.163:0/3398552114' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2026-02-13T10:39:19.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:18 trial121 ceph-mon[19159]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2026-02-13T10:39:19.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:18 trial121 ceph-mon[19159]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-13T10:39:19.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:18 trial163 ceph-mon[24184]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:19.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:18 trial163 ceph-mon[24184]: from='client.? 10.20.193.163:0/3398552114' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2026-02-13T10:39:19.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:18 trial163 ceph-mon[24184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2026-02-13T10:39:19.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:18 trial163 ceph-mon[24184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-13T10:39:19.326 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:19.431 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/2321196274 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e410a8c0 msgr2=0x7fc6e410aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/2321196274 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e410a8c0 0x7fc6e410aca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80099b0 tx=0x7fc6d802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/2321196274 shutdown_connections 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/2321196274 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc6e41048c0 0x7fc6e4104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/2321196274 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e410a8c0 0x7fc6e410aca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/2321196274 >> 10.20.193.121:0/2321196274 conn(0x7fc6e41005c0 msgr2=0x7fc6e41029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/2321196274 shutdown_connections 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.428+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/2321196274 wait complete. 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 Processor -- start 2026-02-13T10:39:19.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 -- start start 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e41048c0 0x7fc6e41a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc6e410a8c0 0x7fc6e41a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e4111c20 con 0x7fc6e41048c0 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e4111aa0 con 0x7fc6e410a8c0 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e2575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e41048c0 0x7fc6e41a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e2575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e41048c0 0x7fc6e41a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50322/0 (socket says 10.20.193.121:50322) 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e2575640 1 -- 10.20.193.121:0/342740109 learned_addr learned my addr 10.20.193.121:0/342740109 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e2575640 1 -- 10.20.193.121:0/342740109 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc6e410a8c0 msgr2=0x7fc6e41a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e2575640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc6e410a8c0 0x7fc6e41a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e2575640 1 -- 10.20.193.121:0/342740109 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e419c770 con 0x7fc6e41048c0 2026-02-13T10:39:19.434 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e2575640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e41048c0 0x7fc6e41a26a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009980 tx=0x7fc6d8002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:19.434 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6cb7fe640 1 -- 10.20.193.121:0/342740109 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fc6d8004420 con 0x7fc6e41048c0 2026-02-13T10:39:19.434 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6cb7fe640 1 -- 10.20.193.121:0/342740109 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fc6d8038c00 con 0x7fc6e41048c0 2026-02-13T10:39:19.434 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e419ca00 con 0x7fc6e41048c0 2026-02-13T10:39:19.434 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6cb7fe640 1 -- 10.20.193.121:0/342740109 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fc6d8041ab0 con 0x7fc6e41048c0 2026-02-13T10:39:19.434 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.429+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e419cf30 con 0x7fc6e41048c0 2026-02-13T10:39:19.434 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.430+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e4106000 con 0x7fc6e41048c0 2026-02-13T10:39:19.435 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.431+0000 7fc6cb7fe640 1 -- 10.20.193.121:0/342740109 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fc6d8038770 con 0x7fc6e41048c0 2026-02-13T10:39:19.435 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.431+0000 7fc6cb7fe640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc6bc077550 0x7fc6bc079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:19.435 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.431+0000 7fc6e1d74640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc6bc077550 0x7fc6bc079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:19.435 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.431+0000 7fc6cb7fe640 1 -- 10.20.193.121:0/342740109 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7fc6d803e070 con 0x7fc6e41048c0 2026-02-13T10:39:19.435 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.431+0000 7fc6e1d74640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc6bc077550 0x7fc6bc079a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6e419dfc0 tx=0x7fc6cc005d70 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:19.436 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.433+0000 7fc6cb7fe640 1 -- 10.20.193.121:0/342740109 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc6d8049340 con 0x7fc6e41048c0 2026-02-13T10:39:19.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.536+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc6e419d1e0 con 0x7fc6e41048c0 2026-02-13T10:39:19.539 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.538+0000 7fc6cb7fe640 1 -- 10.20.193.121:0/342740109 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) ==== 74+0+192111 (secure 0 0 0) 0x7fc6d808bc20 con 0x7fc6e41048c0 2026-02-13T10:39:19.541 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:19.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc6bc077550 msgr2=0x7fc6bc079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc6bc077550 0x7fc6bc079a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6e419dfc0 tx=0x7fc6cc005d70 comp rx=0 tx=0).stop 2026-02-13T10:39:19.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e41048c0 msgr2=0x7fc6e41a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e41048c0 0x7fc6e41a26a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009980 tx=0x7fc6d8002e20 comp rx=0 tx=0).stop 2026-02-13T10:39:19.543 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 shutdown_connections 2026-02-13T10:39:19.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fc6bc077550 0x7fc6bc079a10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fc6e410a8c0 0x7fc6e41a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 --2- 10.20.193.121:0/342740109 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fc6e41048c0 0x7fc6e41a26a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 >> 10.20.193.121:0/342740109 conn(0x7fc6e41005c0 msgr2=0x7fc6e410ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:19.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 shutdown_connections 2026-02-13T10:39:19.544 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.539+0000 7fc6e8ca9640 1 -- 10.20.193.121:0/342740109 wait complete. 2026-02-13T10:39:19.579 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":19,"flags":0,"active_gid":14203,"active_name":"trial121.tkoihk","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6800","nonce":2058604417},{"type":"v1","addr":"10.20.193.121:6801","nonce":2058604417}]},"active_addr":"10.20.193.121:6801/2058604417","active_change":"2026-02-13T10:37:45.544649+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14222,"name":"trial163.xfyaeo","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://10.20.193.121:8443/","prometheus":"http://10.20.193.121:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.121:0","nonce":2981153915}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.121:0","nonce":3670439875}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.121:0","nonce":2268926261}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.121:0","nonce":3570424693}]}]} 2026-02-13T10:39:19.586 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2026-02-13T10:39:19.586 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2026-02-13T10:39:19.587 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T10:39:19.872 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:19.976 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- 10.20.193.121:0/3109126289 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077f40 msgr2=0x7f4598113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.976 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 --2- 10.20.193.121:0/3109126289 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077f40 0x7f4598113220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f45800099b0 tx=0x7f458002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:19.976 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- 10.20.193.121:0/3109126289 shutdown_connections 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 --2- 10.20.193.121:0/3109126289 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077f40 0x7f4598113220 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 --2- 10.20.193.121:0/3109126289 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4598077620 0x7f4598077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- 10.20.193.121:0/3109126289 >> 10.20.193.121:0/3109126289 conn(0x7f45981005c0 msgr2=0x7f45981029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- 10.20.193.121:0/3109126289 shutdown_connections 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- 10.20.193.121:0/3109126289 wait complete. 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 Processor -- start 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- start start 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077620 0x7f45981a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4598077f40 0x7f45981a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4598116170 con 0x7f4598077620 2026-02-13T10:39:19.977 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459ec49640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4598115ff0 con 0x7f4598077f40 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459c9be640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077620 0x7f45981a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459c9be640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077620 0x7f45981a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50330/0 (socket says 10.20.193.121:50330) 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.973+0000 7f459c9be640 1 -- 10.20.193.121:0/2695538318 learned_addr learned my addr 10.20.193.121:0/2695538318 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f458ffff640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4598077f40 0x7f45981a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f459c9be640 1 -- 10.20.193.121:0/2695538318 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4598077f40 msgr2=0x7f45981a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f459c9be640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4598077f40 0x7f45981a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f459c9be640 1 -- 10.20.193.121:0/2695538318 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45981a56e0 con 0x7f4598077620 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f459c9be640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077620 0x7f45981a0e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f458800c8f0 tx=0x7f458800cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f458dffb640 1 -- 10.20.193.121:0/2695538318 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4588007c30 con 0x7f4598077620 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f458dffb640 1 -- 10.20.193.121:0/2695538318 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f4588007dd0 con 0x7f4598077620 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45981a59d0 con 0x7f4598077620 2026-02-13T10:39:19.978 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f458dffb640 1 -- 10.20.193.121:0/2695538318 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4588018840 con 0x7f4598077620 2026-02-13T10:39:19.979 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45981a5f60 con 0x7f4598077620 2026-02-13T10:39:19.979 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.974+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4598078d60 con 0x7f4598077620 2026-02-13T10:39:19.980 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.975+0000 7f458dffb640 1 -- 10.20.193.121:0/2695538318 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f45880189e0 con 0x7f4598077620 2026-02-13T10:39:19.980 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.975+0000 7f458dffb640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4574077550 0x7f4574079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:19.980 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.975+0000 7f458dffb640 1 -- 10.20.193.121:0/2695538318 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f4588099b20 con 0x7f4598077620 2026-02-13T10:39:19.980 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.975+0000 7f458ffff640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4574077550 0x7f4574079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:19.980 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.976+0000 7f458ffff640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4574077550 0x7f4574079a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4598100ba0 tx=0x7f458003a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:19.980 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:19.977+0000 7f458dffb640 1 -- 10.20.193.121:0/2695538318 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4588014030 con 0x7f4598077620 2026-02-13T10:39:20.061 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.060+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45981a6270 con 0x7f4598077620 2026-02-13T10:39:20.062 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.060+0000 7f458dffb640 1 -- 10.20.193.121:0/2695538318 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+11451 (secure 0 0 0) 0x7f458801f300 con 0x7f4598077620 2026-02-13T10:39:20.063 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:20.063 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":37,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","created":"2026-02-13T10:37:09.311059+0000","modified":"2026-02-13T10:39:16.864364+0000","last_up_change":"2026-02-13T10:39:15.860548+0000","last_in_change":"2026-02-13T10:38:59.652055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T10:38:43.586630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15b6509f-b28f-4bda-bbdc-c0c0b3074fca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6802","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6803","nonce":1486620450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6804","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6805","nonce":1486620450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6808","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6809","nonce":1486620450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6806","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6807","nonce":1486620450}]},"public_addr":"10.20.193.121:6803/1486620450","cluster_addr":"10.20.193.121:6805/1486620450","heartbeat_back_addr":"10.20.193.121:6809/1486620450","heartbeat_front_addr":"10.20.193.121:6807/1486620450","state":["exists","up"]},{"osd":1,"uuid":"c3e08c6d-c2cb-4d19-988c-7ec1d5d70aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6810","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6811","nonce":4000550628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6812","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6813","nonce":4000550628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6816","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6817","nonce":4000550628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6814","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6815","nonce":4000550628}]},"public_addr":"10.20.193.121:6811/4000550628","cluster_addr":"10.20.193.121:6813/4000550628","heartbeat_back_addr":"10.20.193.121:6817/4000550628","heartbeat_front_addr":"10.20.193.121:6815/4000550628","state":["exists","up"]},{"osd":2,"uuid":"3aad7001-dd1d-4ab3-bee7-17801f2b1563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6818","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6819","nonce":3599178802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6820","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6821","nonce":3599178802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6824","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6825","nonce":3599178802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6822","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6823","nonce":3599178802}]},"public_addr":"10.20.193.121:6819/3599178802","cluster_addr":"10.20.193.121:6821/3599178802","heartbeat_back_addr":"10.20.193.121:6825/3599178802","heartbeat_front_addr":"10.20.193.121:6823/3599178802","state":["exists","up"]},{"osd":3,"uuid":"09d5bd71-44e9-4c3f-a3b6-f15074495bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6800","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6801","nonce":2365834735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6802","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6803","nonce":2365834735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6806","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6807","nonce":2365834735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6804","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6805","nonce":2365834735}]},"public_addr":"10.20.193.163:6801/2365834735","cluster_addr":"10.20.193.163:6803/2365834735","heartbeat_back_addr":"10.20.193.163:6807/2365834735","heartbeat_front_addr":"10.20.193.163:6805/2365834735","state":["exists","up"]},{"osd":4,"uuid":"7d965c3c-65dd-4fd9-a1ee-996241c8d2e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6808","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6809","nonce":3393667471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6810","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6811","nonce":3393667471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6814","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6815","nonce":3393667471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6812","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6813","nonce":3393667471}]},"public_addr":"10.20.193.163:6809/3393667471","cluster_addr":"10.20.193.163:6811/3393667471","heartbeat_back_addr":"10.20.193.163:6815/3393667471","heartbeat_front_addr":"10.20.193.163:6813/3393667471","state":["exists","up"]},{"osd":5,"uuid":"d5e8a5b3-c3d6-4c49-97ce-8641644d24cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6816","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6817","nonce":2139375176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6818","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6819","nonce":2139375176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6822","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6823","nonce":2139375176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6820","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6821","nonce":2139375176}]},"public_addr":"10.20.193.163:6817/2139375176","cluster_addr":"10.20.193.163:6819/2139375176","heartbeat_back_addr":"10.20.193.163:6823/2139375176","heartbeat_front_addr":"10.20.193.163:6821/2139375176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:19.148789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:30.586490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:41.413339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:52.330028+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:39:03.516006+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:39:14.506111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.121:0/1986687891":"2026-02-14T10:37:45.544561+0000","10.20.193.121:0/243024684":"2026-02-14T10:37:19.150822+0000","10.20.193.121:6801/232220030":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6801/3394068414":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/1469338567":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/1306451008":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/1699876434":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/3836430930":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/1389694817":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/2646306236":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6801/977129689":"2026-02-14T10:37:28.590631+0000","10.20.193.121:6800/977129689":"2026-02-14T10:37:28.590631+0000","10.20.193.121:6800/3394068414":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/3763370872":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6800/232220030":"2026-02-14T10:37:45.544561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T10:39:20.063 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4574077550 msgr2=0x7f4574079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4574077550 0x7f4574079a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4598100ba0 tx=0x7f458003a040 comp rx=0 tx=0).stop 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077620 msgr2=0x7f45981a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077620 0x7f45981a0e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f458800c8f0 tx=0x7f458800cdc0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 shutdown_connections 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4574077550 0x7f4574079a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4598077f40 0x7f45981a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 --2- 10.20.193.121:0/2695538318 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4598077620 0x7f45981a0e10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 >> 10.20.193.121:0/2695538318 conn(0x7f45981005c0 msgr2=0x7f45981012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 shutdown_connections 2026-02-13T10:39:20.064 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.061+0000 7f459ec49640 1 -- 10.20.193.121:0/2695538318 wait complete. 2026-02-13T10:39:20.102 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2026-02-13T10:39:20.102 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T10:39:20.124 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:19 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3319502972' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2026-02-13T10:39:20.125 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:19 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/342740109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2026-02-13T10:39:20.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:19 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3319502972' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2026-02-13T10:39:20.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:19 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/342740109' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2026-02-13T10:39:20.388 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:20.496 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- 10.20.193.121:0/3817095114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 msgr2=0x7f9e0c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 --2- 10.20.193.121:0/3817095114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c104d20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0099b0 tx=0x7f9dfc02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- 10.20.193.121:0/3817095114 shutdown_connections 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 --2- 10.20.193.121:0/3817095114 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c104d20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 --2- 10.20.193.121:0/3817095114 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e0c10a8c0 0x7f9e0c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- 10.20.193.121:0/3817095114 >> 10.20.193.121:0/3817095114 conn(0x7f9e0c1005c0 msgr2=0x7f9e0c1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- 10.20.193.121:0/3817095114 shutdown_connections 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- 10.20.193.121:0/3817095114 wait complete. 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 Processor -- start 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- start start 2026-02-13T10:39:20.497 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c1a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e0c10a8c0 0x7f9e0c1a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c111c20 con 0x7f9e0c1048c0 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e1287a640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c111aa0 con 0x7f9e0c10a8c0 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e0bfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c1a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e0bfff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c1a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50342/0 (socket says 10.20.193.121:50342) 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e0bfff640 1 -- 10.20.193.121:0/421424610 learned_addr learned my addr 10.20.193.121:0/421424610 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e0b7fe640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e0c10a8c0 0x7f9e0c1a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e0bfff640 1 -- 10.20.193.121:0/421424610 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e0c10a8c0 msgr2=0x7f9e0c1a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e0bfff640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e0c10a8c0 0x7f9e0c1a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.493+0000 7f9e0bfff640 1 -- 10.20.193.121:0/421424610 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c19c730 con 0x7f9e0c1048c0 2026-02-13T10:39:20.498 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.494+0000 7f9e0bfff640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c1a2660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9df800d8d0 tx=0x7f9df800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:20.499 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.494+0000 7f9e097fa640 1 -- 10.20.193.121:0/421424610 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9df8014070 con 0x7f9e0c1048c0 2026-02-13T10:39:20.499 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.494+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c19ca20 con 0x7f9e0c1048c0 2026-02-13T10:39:20.499 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.494+0000 7f9e097fa640 1 -- 10.20.193.121:0/421424610 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f9df80044a0 con 0x7f9e0c1048c0 2026-02-13T10:39:20.499 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.494+0000 7f9e097fa640 1 -- 10.20.193.121:0/421424610 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9df8002e60 con 0x7f9e0c1048c0 2026-02-13T10:39:20.499 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.494+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c19cf80 con 0x7f9e0c1048c0 2026-02-13T10:39:20.499 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.494+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e0c106000 con 0x7f9e0c1048c0 2026-02-13T10:39:20.499 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.495+0000 7f9e097fa640 1 -- 10.20.193.121:0/421424610 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f9df800b840 con 0x7f9e0c1048c0 2026-02-13T10:39:20.500 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.495+0000 7f9e097fa640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9de00775a0 0x7f9de0079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:20.500 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.495+0000 7f9e097fa640 1 -- 10.20.193.121:0/421424610 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f9df8099710 con 0x7f9e0c1048c0 2026-02-13T10:39:20.500 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.496+0000 7f9e0b7fe640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9de00775a0 0x7f9de0079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:20.500 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.496+0000 7f9e0b7fe640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9de00775a0 0x7f9de0079a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0097c0 tx=0x7f9dfc03a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:20.500 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.497+0000 7f9e097fa640 1 -- 10.20.193.121:0/421424610 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9df8066b80 con 0x7f9e0c1048c0 2026-02-13T10:39:20.582 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.581+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e0c104d20 con 0x7f9e0c1048c0 2026-02-13T10:39:20.583 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.581+0000 7f9e097fa640 1 -- 10.20.193.121:0/421424610 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+11451 (secure 0 0 0) 0x7f9df806ba30 con 0x7f9e0c1048c0 2026-02-13T10:39:20.583 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:20.583 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":37,"fsid":"e9ef2631-08c7-11f1-a2b3-d404e6e7d460","created":"2026-02-13T10:37:09.311059+0000","modified":"2026-02-13T10:39:16.864364+0000","last_up_change":"2026-02-13T10:39:15.860548+0000","last_in_change":"2026-02-13T10:38:59.652055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T10:38:43.586630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15b6509f-b28f-4bda-bbdc-c0c0b3074fca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6802","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6803","nonce":1486620450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6804","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6805","nonce":1486620450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6808","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6809","nonce":1486620450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6806","nonce":1486620450},{"type":"v1","addr":"10.20.193.121:6807","nonce":1486620450}]},"public_addr":"10.20.193.121:6803/1486620450","cluster_addr":"10.20.193.121:6805/1486620450","heartbeat_back_addr":"10.20.193.121:6809/1486620450","heartbeat_front_addr":"10.20.193.121:6807/1486620450","state":["exists","up"]},{"osd":1,"uuid":"c3e08c6d-c2cb-4d19-988c-7ec1d5d70aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6810","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6811","nonce":4000550628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6812","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6813","nonce":4000550628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6816","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6817","nonce":4000550628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6814","nonce":4000550628},{"type":"v1","addr":"10.20.193.121:6815","nonce":4000550628}]},"public_addr":"10.20.193.121:6811/4000550628","cluster_addr":"10.20.193.121:6813/4000550628","heartbeat_back_addr":"10.20.193.121:6817/4000550628","heartbeat_front_addr":"10.20.193.121:6815/4000550628","state":["exists","up"]},{"osd":2,"uuid":"3aad7001-dd1d-4ab3-bee7-17801f2b1563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6818","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6819","nonce":3599178802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6820","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6821","nonce":3599178802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6824","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6825","nonce":3599178802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6822","nonce":3599178802},{"type":"v1","addr":"10.20.193.121:6823","nonce":3599178802}]},"public_addr":"10.20.193.121:6819/3599178802","cluster_addr":"10.20.193.121:6821/3599178802","heartbeat_back_addr":"10.20.193.121:6825/3599178802","heartbeat_front_addr":"10.20.193.121:6823/3599178802","state":["exists","up"]},{"osd":3,"uuid":"09d5bd71-44e9-4c3f-a3b6-f15074495bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6800","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6801","nonce":2365834735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6802","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6803","nonce":2365834735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6806","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6807","nonce":2365834735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6804","nonce":2365834735},{"type":"v1","addr":"10.20.193.163:6805","nonce":2365834735}]},"public_addr":"10.20.193.163:6801/2365834735","cluster_addr":"10.20.193.163:6803/2365834735","heartbeat_back_addr":"10.20.193.163:6807/2365834735","heartbeat_front_addr":"10.20.193.163:6805/2365834735","state":["exists","up"]},{"osd":4,"uuid":"7d965c3c-65dd-4fd9-a1ee-996241c8d2e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6808","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6809","nonce":3393667471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6810","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6811","nonce":3393667471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6814","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6815","nonce":3393667471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6812","nonce":3393667471},{"type":"v1","addr":"10.20.193.163:6813","nonce":3393667471}]},"public_addr":"10.20.193.163:6809/3393667471","cluster_addr":"10.20.193.163:6811/3393667471","heartbeat_back_addr":"10.20.193.163:6815/3393667471","heartbeat_front_addr":"10.20.193.163:6813/3393667471","state":["exists","up"]},{"osd":5,"uuid":"d5e8a5b3-c3d6-4c49-97ce-8641644d24cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6816","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6817","nonce":2139375176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6818","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6819","nonce":2139375176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6822","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6823","nonce":2139375176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6820","nonce":2139375176},{"type":"v1","addr":"10.20.193.163:6821","nonce":2139375176}]},"public_addr":"10.20.193.163:6817/2139375176","cluster_addr":"10.20.193.163:6819/2139375176","heartbeat_back_addr":"10.20.193.163:6823/2139375176","heartbeat_front_addr":"10.20.193.163:6821/2139375176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:19.148789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:30.586490+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:41.413339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:38:52.330028+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:39:03.516006+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T10:39:14.506111+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.121:0/1986687891":"2026-02-14T10:37:45.544561+0000","10.20.193.121:0/243024684":"2026-02-14T10:37:19.150822+0000","10.20.193.121:6801/232220030":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6801/3394068414":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/1469338567":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/1306451008":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/1699876434":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/3836430930":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/1389694817":"2026-02-14T10:37:28.590631+0000","10.20.193.121:0/2646306236":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6801/977129689":"2026-02-14T10:37:28.590631+0000","10.20.193.121:6800/977129689":"2026-02-14T10:37:28.590631+0000","10.20.193.121:6800/3394068414":"2026-02-14T10:37:19.150822+0000","10.20.193.121:0/3763370872":"2026-02-14T10:37:45.544561+0000","10.20.193.121:6800/232220030":"2026-02-14T10:37:45.544561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T10:39:20.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9de00775a0 msgr2=0x7f9de0079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:20.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9de00775a0 0x7f9de0079a60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0097c0 tx=0x7f9dfc03a040 comp rx=0 tx=0).stop 2026-02-13T10:39:20.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 msgr2=0x7f9e0c1a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c1a2660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9df800d8d0 tx=0x7f9df800dda0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 shutdown_connections 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9de00775a0 0x7f9de0079a60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9e0c10a8c0 0x7f9e0c1a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 --2- 10.20.193.121:0/421424610 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9e0c1048c0 0x7f9e0c1a2660 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 >> 10.20.193.121:0/421424610 conn(0x7f9e0c1005c0 msgr2=0x7f9e0c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 shutdown_connections 2026-02-13T10:39:20.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:20.582+0000 7f9e1287a640 1 -- 10.20.193.121:0/421424610 wait complete. 2026-02-13T10:39:20.620 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph tell osd.0 flush_pg_stats 2026-02-13T10:39:20.621 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph tell osd.1 flush_pg_stats 2026-02-13T10:39:20.621 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph tell osd.2 flush_pg_stats 2026-02-13T10:39:20.622 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph tell osd.3 flush_pg_stats 2026-02-13T10:39:20.622 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph tell osd.4 flush_pg_stats 2026-02-13T10:39:20.623 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph tell osd.5 flush_pg_stats 2026-02-13T10:39:20.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:20 trial121 ceph-mon[19159]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:20.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:20 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2695538318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2026-02-13T10:39:20.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:20 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/421424610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2026-02-13T10:39:21.024 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.024 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.025 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.026 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.026 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.026 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.138 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f6aff9ab640 1 -- 10.20.193.121:0/4239101575 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 msgr2=0x7f6af810aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.138 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/4239101575 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af810aca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80099b0 tx=0x7f6ae802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.138 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f6aff9ab640 1 -- 10.20.193.121:0/4239101575 shutdown_connections 2026-02-13T10:39:21.139 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/4239101575 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6af81048c0 0x7f6af8104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.139 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/4239101575 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af810aca0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.139 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f6aff9ab640 1 -- 10.20.193.121:0/4239101575 >> 10.20.193.121:0/4239101575 conn(0x7f6af81005c0 msgr2=0x7f6af81029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.139 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f6aff9ab640 1 -- 10.20.193.121:0/4239101575 shutdown_connections 2026-02-13T10:39:21.139 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.136+0000 7f4868f95640 1 -- 10.20.193.121:0/1453542413 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 msgr2=0x7f4864104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.139 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 --2- 10.20.193.121:0/1453542413 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f4864104d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f484c0099b0 tx=0x7f484c02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.140 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f6aff9ab640 1 -- 10.20.193.121:0/4239101575 wait complete. 2026-02-13T10:39:21.140 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 -- 10.20.193.121:0/1453542413 shutdown_connections 2026-02-13T10:39:21.141 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 --2- 10.20.193.121:0/1453542413 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f4864104d10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.141 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 --2- 10.20.193.121:0/1453542413 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f486410a8b0 0x7f486410ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.141 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 -- 10.20.193.121:0/1453542413 >> 10.20.193.121:0/1453542413 conn(0x7f4864100590 msgr2=0x7f48641029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.141 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 -- 10.20.193.121:0/1453542413 shutdown_connections 2026-02-13T10:39:21.141 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 -- 10.20.193.121:0/1453542413 wait complete. 2026-02-13T10:39:21.141 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 -- 10.20.193.121:0/74479889 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 msgr2=0x7f974c1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.142 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 --2- 10.20.193.121:0/74479889 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c1019a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f97340099b0 tx=0x7f973402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.142 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 Processor -- start 2026-02-13T10:39:21.142 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 -- 10.20.193.121:0/74479889 shutdown_connections 2026-02-13T10:39:21.143 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 --2- 10.20.193.121:0/74479889 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c1019a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.143 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 --2- 10.20.193.121:0/74479889 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f974c1075f0 0x7f974c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.143 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 -- 10.20.193.121:0/74479889 >> 10.20.193.121:0/74479889 conn(0x7f974c0fd220 msgr2=0x7f974c0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.143 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 -- 10.20.193.121:0/74479889 shutdown_connections 2026-02-13T10:39:21.143 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f9750aba640 1 -- 10.20.193.121:0/74479889 wait complete. 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 -- start start 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f48641a2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f486410a8b0 0x7f48641a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4864111c30 con 0x7f48641048b0 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4868f95640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4864111ab0 con 0x7f486410a8b0 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4862575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f48641a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4862575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f48641a2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50376/0 (socket says 10.20.193.121:50376) 2026-02-13T10:39:21.144 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.137+0000 7f4862575640 1 -- 10.20.193.121:0/1547340457 learned_addr learned my addr 10.20.193.121:0/1547340457 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.145 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f9750aba640 1 Processor -- start 2026-02-13T10:39:21.145 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f4862575640 1 -- 10.20.193.121:0/1547340457 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f486410a8b0 msgr2=0x7f48641a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.145 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f4862575640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f486410a8b0 0x7f48641a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6aff9ab640 1 Processor -- start 2026-02-13T10:39:21.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f4862575640 1 -- 10.20.193.121:0/1547340457 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486419c740 con 0x7f48641048b0 2026-02-13T10:39:21.146 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f4862575640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f48641a2670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f485800d900 tx=0x7f485800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6aff9ab640 1 -- start start 2026-02-13T10:39:21.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 -- 10.20.193.121:0/892851379 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 msgr2=0x7f1758113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 --2- 10.20.193.121:0/892851379 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 0x7f1758113220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f174c0099b0 tx=0x7f174c02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.147 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 -- 10.20.193.121:0/892851379 shutdown_connections 2026-02-13T10:39:21.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 --2- 10.20.193.121:0/892851379 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 0x7f1758113220 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 --2- 10.20.193.121:0/892851379 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f1758077620 0x7f1758077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 -- 10.20.193.121:0/892851379 >> 10.20.193.121:0/892851379 conn(0x7f17581005c0 msgr2=0x7f17581029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 -- 10.20.193.121:0/892851379 shutdown_connections 2026-02-13T10:39:21.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f9750aba640 1 -- start start 2026-02-13T10:39:21.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f9750aba640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.148 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f9750aba640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f974c1075f0 0x7f974c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f9750aba640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10e8b0 con 0x7f974c101540 2026-02-13T10:39:21.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f9750aba640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10e730 con 0x7f974c1075f0 2026-02-13T10:39:21.149 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 -- 10.20.193.121:0/892851379 wait complete. 2026-02-13T10:39:21.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4858014070 con 0x7f48641048b0 2026-02-13T10:39:21.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f48580044a0 con 0x7f48641048b0 2026-02-13T10:39:21.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486419ca30 con 0x7f48641048b0 2026-02-13T10:39:21.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4858005020 con 0x7f48641048b0 2026-02-13T10:39:21.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486419ced0 con 0x7f48641048b0 2026-02-13T10:39:21.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f974a575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.150 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f974a575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50384/0 (socket says 10.20.193.121:50384) 2026-02-13T10:39:21.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f974a575640 1 -- 10.20.193.121:0/810938439 learned_addr learned my addr 10.20.193.121:0/810938439 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f9749d74640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f974c1075f0 0x7f974c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6aff9ab640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6af81048c0 0x7f6af81a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6aff9ab640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af81a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6aff9ab640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8111c20 con 0x7f6af810a8c0 2026-02-13T10:39:21.151 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6aff9ab640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8111aa0 con 0x7f6af81048c0 2026-02-13T10:39:21.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afcf1f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af81a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afcf1f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af81a2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50392/0 (socket says 10.20.193.121:50392) 2026-02-13T10:39:21.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afcf1f640 1 -- 10.20.193.121:0/2423503691 learned_addr learned my addr 10.20.193.121:0/2423503691 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 Processor -- start 2026-02-13T10:39:21.152 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f175fa9d640 1 -- start start 2026-02-13T10:39:21.153 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afd720640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6af81048c0 0x7f6af81a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.153 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f974a575640 1 -- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f974c1075f0 msgr2=0x7f974c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.153 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f974a575640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f974c1075f0 0x7f974c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.153 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f974a575640 1 -- 10.20.193.121:0/810938439 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c19eed0 con 0x7f974c101540 2026-02-13T10:39:21.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afcf1f640 1 -- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6af81048c0 msgr2=0x7f6af81a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afcf1f640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6af81048c0 0x7f6af81a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f486419d4f0 con 0x7f48641048b0 2026-02-13T10:39:21.154 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f974a575640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c19a600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f974000d900 tx=0x7f974000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.155 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afcf1f640 1 -- 10.20.193.121:0/2423503691 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af819c6d0 con 0x7f6af810a8c0 2026-02-13T10:39:21.155 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.138+0000 7f6afd720640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6af81048c0 0x7f6af81a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T10:39:21.155 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9740014070 con 0x7f974c101540 2026-02-13T10:39:21.156 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f6afcf1f640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af81a2b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00c910 tx=0x7f6aec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.156 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f97400044a0 con 0x7f974c101540 2026-02-13T10:39:21.156 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9740005020 con 0x7f974c101540 2026-02-13T10:39:21.157 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175fa9d640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f1758077620 0x7f175806ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.157 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175fa9d640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 0x7f1758070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.157 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c19f1c0 con 0x7f974c101540 2026-02-13T10:39:21.157 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c19f7d0 con 0x7f974c101540 2026-02-13T10:39:21.157 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 -- 10.20.193.121:0/3594231305 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2494100ec0 msgr2=0x7f2494113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.157 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 --2- 10.20.193.121:0/3594231305 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2494100ec0 0x7f2494113160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f247c0099b0 tx=0x7f247c02f2b0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.158 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175fa9d640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1758116150 con 0x7f1758077f40 2026-02-13T10:39:21.158 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d812640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f1758077620 0x7f175806ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.158 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d812640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f1758077620 0x7f175806ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.163:3300/0 says I am v2:10.20.193.121:54570/0 (socket says 10.20.193.121:54570) 2026-02-13T10:39:21.159 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f974c19b280 con 0x7f974c101540 2026-02-13T10:39:21.159 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d812640 1 -- 10.20.193.121:0/265317554 learned_addr learned my addr 10.20.193.121:0/265317554 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.159 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1758115fd0 con 0x7f1758077620 2026-02-13T10:39:21.159 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d011640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 0x7f1758070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 -- 10.20.193.121:0/3594231305 shutdown_connections 2026-02-13T10:39:21.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 --2- 10.20.193.121:0/3594231305 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2494100ec0 0x7f2494113160 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 --2- 10.20.193.121:0/3594231305 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f24941005a0 0x7f2494100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 -- 10.20.193.121:0/3594231305 >> 10.20.193.121:0/3594231305 conn(0x7f24940fc4f0 msgr2=0x7f24940fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 -- 10.20.193.121:0/3594231305 shutdown_connections 2026-02-13T10:39:21.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d011640 1 -- 10.20.193.121:0/265317554 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f1758077620 msgr2=0x7f175806ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.161 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d011640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f1758077620 0x7f175806ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.161 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f48580047c0 con 0x7f48641048b0 2026-02-13T10:39:21.161 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d011640 1 -- 10.20.193.121:0/265317554 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1758070a40 con 0x7f1758077f40 2026-02-13T10:39:21.161 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f175d011640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 0x7f1758070500 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f174c0099b0 tx=0x7f174c0028f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.162 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f2499067640 1 -- 10.20.193.121:0/3594231305 wait complete. 2026-02-13T10:39:21.162 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f484b7fe640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f483c0776c0 0x7f483c079b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.162 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 Processor -- start 2026-02-13T10:39:21.163 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f4858099700 con 0x7f48641048b0 2026-02-13T10:39:21.163 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.139+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f174c004260 con 0x7f1758077f40 2026-02-13T10:39:21.163 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f174c004400 con 0x7f1758077f40 2026-02-13T10:39:21.163 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f174c041910 con 0x7f1758077f40 2026-02-13T10:39:21.163 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1758070cd0 con 0x7f1758077f40 2026-02-13T10:39:21.163 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581b3b90 con 0x7f1758077f40 2026-02-13T10:39:21.164 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f484b7fe640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] conn(0x7f483c07d4b0 0x7f483c07f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.164 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 --> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4858013ec0 con 0x7f483c07d4b0 2026-02-13T10:39:21.164 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4858099af0 con 0x7f48641048b0 2026-02-13T10:39:21.164 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6aec007c30 con 0x7f6af810a8c0 2026-02-13T10:39:21.164 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f6aec007dd0 con 0x7f6af810a8c0 2026-02-13T10:39:21.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af819c9c0 con 0x7f6af810a8c0 2026-02-13T10:39:21.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f4862d76640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] conn(0x7f483c07d4b0 0x7f483c07f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f4861d74640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f483c0776c0 0x7f483c079b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 -- start start 2026-02-13T10:39:21.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f24941005a0 0x7f249410cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.166 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6aec00f450 con 0x7f6af810a8c0 2026-02-13T10:39:21.166 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af819cec0 con 0x7f6af810a8c0 2026-02-13T10:39:21.166 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1758063390 con 0x7f1758077f40 2026-02-13T10:39:21.167 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f97400047c0 con 0x7f974c101540 2026-02-13T10:39:21.167 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f4862d76640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] conn(0x7f483c07d4b0 0x7f483c07f8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.167 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f97337fe640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f97240776c0 0x7f9724079b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2494100ec0 0x7f249410d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2494116130 con 0x7f24941005a0 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2494115fb0 con 0x7f2494100ec0 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2492d76640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f24941005a0 0x7f249410cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2492d76640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f24941005a0 0x7f249410cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50420/0 (socket says 10.20.193.121:50420) 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2492d76640 1 -- 10.20.193.121:0/4179735902 learned_addr learned my addr 10.20.193.121:0/4179735902 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2492d76640 1 -- 10.20.193.121:0/4179735902 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2494100ec0 msgr2=0x7f249410d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2492d76640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2494100ec0 0x7f249410d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2492d76640 1 -- 10.20.193.121:0/4179735902 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249410d980 con 0x7f24941005a0 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2492d76640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f24941005a0 0x7f249410cf00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f248800e9b0 tx=0x7f248800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.168 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f248800cbe0 con 0x7f24941005a0 2026-02-13T10:39:21.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 -- 10.20.193.121:0/4179735902 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249410dc70 con 0x7f24941005a0 2026-02-13T10:39:21.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2499067640 1 -- 10.20.193.121:0/4179735902 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941b3bc0 con 0x7f24941005a0 2026-02-13T10:39:21.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f2488004560 con 0x7f24941005a0 2026-02-13T10:39:21.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.140+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f24880048a0 con 0x7f24941005a0 2026-02-13T10:39:21.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f9749d74640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f97240776c0 0x7f9724079b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6af819d540 con 0x7f6af810a8c0 2026-02-13T10:39:21.170 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f174c041ab0 con 0x7f1758077f40 2026-02-13T10:39:21.170 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f9740099610 con 0x7f974c101540 2026-02-13T10:39:21.170 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f97337fe640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] conn(0x7f972407d4b0 0x7f972407f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.171 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 --> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9740012e80 con 0x7f972407d4b0 2026-02-13T10:39:21.171 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9740099a00 con 0x7f974c101540 2026-02-13T10:39:21.171 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== osd.1 v2:10.20.193.121:6810/4000550628 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f4858013ec0 con 0x7f483c07d4b0 2026-02-13T10:39:21.171 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f245c001030 con 0x7f24941005a0 2026-02-13T10:39:21.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f9749d74640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f97240776c0 0x7f9724079b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f974c100970 tx=0x7f9734005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f974ad76640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] conn(0x7f972407d4b0 0x7f972407f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f4861d74640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f483c0776c0 0x7f483c079b80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f484c0097c0 tx=0x7f484c005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f24880040d0 con 0x7f24941005a0 2026-02-13T10:39:21.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f1746ffd640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f17340775a0 0x7f1734079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f175d812640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f17340775a0 0x7f1734079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f174c03e070 con 0x7f1758077f40 2026-02-13T10:39:21.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f1746ffd640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] conn(0x7f173407d390 0x7f173407f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 --> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f174c007e80 con 0x7f173407d390 2026-02-13T10:39:21.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f174c0c5050 con 0x7f1758077f40 2026-02-13T10:39:21.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f6aec005000 con 0x7f6af810a8c0 2026-02-13T10:39:21.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f175d812640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f17340775a0 0x7f1734079a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17480059c0 tx=0x7f174800a430 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f175e013640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] conn(0x7f173407d390 0x7f173407f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.175 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f974ad76640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] conn(0x7f972407d4b0 0x7f972407f8d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.175 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f6d576e7640 1 -- 10.20.193.121:0/428101005 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 msgr2=0x7f6d50103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.175 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.141+0000 7f6d576e7640 1 --2- 10.20.193.121:0/428101005 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50103c40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6d400099e0 tx=0x7f6d4002f310 comp rx=0 tx=0).stop 2026-02-13T10:39:21.176 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2473fff640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f24680775a0 0x7f2468079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.176 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f2488099860 con 0x7f24941005a0 2026-02-13T10:39:21.176 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2492575640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f24680775a0 0x7f2468079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.176 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2473fff640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] conn(0x7f246807d390 0x7f246807f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.176 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 --> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f246807fe60 con 0x7f246807d390 2026-02-13T10:39:21.176 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f248809e2a0 con 0x7f24941005a0 2026-02-13T10:39:21.176 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2493577640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] conn(0x7f246807d390 0x7f246807f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.177 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6ae67fc640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6ad40776c0 0x7f6ad4079b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.177 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f6aec0994c0 con 0x7f6af810a8c0 2026-02-13T10:39:21.177 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f175e013640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] conn(0x7f173407d390 0x7f173407f7b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.177 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f2493577640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] conn(0x7f246807d390 0x7f246807f7b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6ae67fc640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] conn(0x7f6ad407d4b0 0x7f6ad407f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 --> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6aec016ec0 con 0x7f6ad407d4b0 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6aec0a0050 con 0x7f6af810a8c0 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6d576e7640 1 -- 10.20.193.121:0/428101005 shutdown_connections 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6d576e7640 1 --2- 10.20.193.121:0/428101005 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6d50104180 0x7f6d5010c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6d576e7640 1 --2- 10.20.193.121:0/428101005 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50103c40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6d576e7640 1 -- 10.20.193.121:0/428101005 >> 10.20.193.121:0/428101005 conn(0x7f6d500fd220 msgr2=0x7f6d500ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6d576e7640 1 -- 10.20.193.121:0/428101005 shutdown_connections 2026-02-13T10:39:21.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6d576e7640 1 -- 10.20.193.121:0/428101005 wait complete. 2026-02-13T10:39:21.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6afdf21640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] conn(0x7f6ad407d4b0 0x7f6ad407f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== osd.0 v2:10.20.193.121:6802/1486620450 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f174c007e80 con 0x7f173407d390 2026-02-13T10:39:21.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== osd.2 v2:10.20.193.121:6818/3599178802 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f246807fe60 con 0x7f246807d390 2026-02-13T10:39:21.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f2492575640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f24680775a0 0x7f2468079a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097c0 tx=0x7f247c004800 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6afd720640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6ad40776c0 0x7f6ad4079b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== osd.4 v2:10.20.193.163:6808/3393667471 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f9740012e80 con 0x7f972407d4b0 2026-02-13T10:39:21.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6afdf21640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] conn(0x7f6ad407d4b0 0x7f6ad407f8d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.142+0000 7f6d576e7640 1 Processor -- start 2026-02-13T10:39:21.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d576e7640 1 -- start start 2026-02-13T10:39:21.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6afd720640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6ad40776c0 0x7f6ad4079b80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005e90 tx=0x7f6ae803a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d576e7640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d576e7640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6d50104180 0x7f6d50101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d576e7640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010fa80 con 0x7f6d50103860 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d576e7640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5010f900 con 0x7f6d50104180 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d5545c640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d5545c640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50428/0 (socket says 10.20.193.121:50428) 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d5545c640 1 -- 10.20.193.121:0/2799957656 learned_addr learned my addr 10.20.193.121:0/2799957656 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d54c5b640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6d50104180 0x7f6d50101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d5545c640 1 -- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6d50104180 msgr2=0x7f6d50101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d5545c640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6d50104180 0x7f6d50101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.143+0000 7f6d5545c640 1 -- 10.20.193.121:0/2799957656 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d50101ec0 con 0x7f6d50103860 2026-02-13T10:39:21.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== osd.5 v2:10.20.193.163:6816/2139375176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f6aec016ec0 con 0x7f6ad407d4b0 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6d5545c640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50101440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002f820 tx=0x7f6d40004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6d40046070 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f6d40038900 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f6d40041890 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d50102150 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d501a6d20 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.144+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d50063350 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f6d40038470 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d3e7fc640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6d2c0775a0 0x7f6d2c079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d54c5b640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6d2c0775a0 0x7f6d2c079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f6d4003e070 con 0x7f6d50103860 2026-02-13T10:39:21.184 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d3e7fc640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] conn(0x7f6d2c07d390 0x7f6d2c07f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 --> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d40007e80 con 0x7f6d2c07d390 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6d400c04e0 con 0x7f6d50103860 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.145+0000 7f6d54c5b640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6d2c0775a0 0x7f6d2c079a60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6d50102ed0 tx=0x7f6d44008040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.146+0000 7f6d55c5d640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] conn(0x7f6d2c07d390 0x7f6d2c07f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.147+0000 7f6d55c5d640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] conn(0x7f6d2c07d390 0x7f6d2c07f7b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.147+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== osd.3 v2:10.20.193.163:6800/2365834735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f6d40007e80 con 0x7f6d2c07d390 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.149+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 --> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4864063310 con 0x7f483c07d4b0 2026-02-13T10:39:21.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.149+0000 7f484b7fe640 1 -- 10.20.193.121:0/1547340457 <== osd.1 v2:10.20.193.121:6810/4000550628 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4864063310 con 0x7f483c07d4b0 2026-02-13T10:39:21.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.149+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 --> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f245c002ed0 con 0x7f246807d390 2026-02-13T10:39:21.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] conn(0x7f483c07d4b0 msgr2=0x7f483c07f8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] conn(0x7f483c07d4b0 0x7f483c07f8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 --> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1758063810 con 0x7f173407d390 2026-02-13T10:39:21.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f483c0776c0 msgr2=0x7f483c079b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f483c0776c0 0x7f483c079b80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f484c0097c0 tx=0x7f484c005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:21.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 msgr2=0x7f48641a2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f48641a2670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f485800d900 tx=0x7f485800ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2473fff640 1 -- 10.20.193.121:0/4179735902 <== osd.2 v2:10.20.193.121:6818/3599178802 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f245c002ed0 con 0x7f246807d390 2026-02-13T10:39:21.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f1746ffd640 1 -- 10.20.193.121:0/265317554 <== osd.0 v2:10.20.193.121:6802/1486620450 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1758063810 con 0x7f173407d390 2026-02-13T10:39:21.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 shutdown_connections 2026-02-13T10:39:21.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6810/4000550628,v1:10.20.193.121:6811/4000550628] conn(0x7f483c07d4b0 0x7f483c07f8d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f483c0776c0 0x7f483c079b80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f486410a8b0 0x7f48641a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 --2- 10.20.193.121:0/1547340457 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f48641048b0 0x7f48641a2670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] conn(0x7f173407d390 msgr2=0x7f173407f7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] conn(0x7f173407d390 0x7f173407f7b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.188 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 --> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f974c1065d0 con 0x7f972407d4b0 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 >> 10.20.193.121:0/1547340457 conn(0x7f4864100590 msgr2=0x7f48641016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 shutdown_connections 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f4868f95640 1 -- 10.20.193.121:0/1547340457 wait complete. 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f17340775a0 msgr2=0x7f1734079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f17340775a0 0x7f1734079a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17480059c0 tx=0x7f174800a430 comp rx=0 tx=0).stop 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 msgr2=0x7f1758070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 0x7f1758070500 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f174c0099b0 tx=0x7f174c0028f0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.189 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] conn(0x7f246807d390 msgr2=0x7f246807f7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] conn(0x7f246807d390 0x7f246807f7b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f24680775a0 msgr2=0x7f2468079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f24680775a0 0x7f2468079a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097c0 tx=0x7f247c004800 comp rx=0 tx=0).stop 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f24941005a0 msgr2=0x7f249410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f24941005a0 0x7f249410cf00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f248800e9b0 tx=0x7f248800ee80 comp rx=0 tx=0).stop 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f97337fe640 1 -- 10.20.193.121:0/810938439 <== osd.4 v2:10.20.193.163:6808/3393667471 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f974c1065d0 con 0x7f972407d4b0 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 shutdown_connections 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6802/1486620450,v1:10.20.193.121:6803/1486620450] conn(0x7f173407d390 0x7f173407f7b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.190 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f17340775a0 0x7f1734079a60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f1758077f40 0x7f1758070500 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 --2- 10.20.193.121:0/265317554 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f1758077620 0x7f175806ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 >> 10.20.193.121:0/265317554 conn(0x7f17581005c0 msgr2=0x7f1758101290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 shutdown_connections 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f175fa9d640 1 -- 10.20.193.121:0/265317554 wait complete. 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 shutdown_connections 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6818/3599178802,v1:10.20.193.121:6819/3599178802] conn(0x7f246807d390 0x7f246807f7b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f24680775a0 0x7f2468079a60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2494100ec0 0x7f249410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 --2- 10.20.193.121:0/4179735902 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f24941005a0 0x7f249410cf00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 >> 10.20.193.121:0/4179735902 conn(0x7f24940fc4f0 msgr2=0x7f24941127d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 shutdown_connections 2026-02-13T10:39:21.191 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.150+0000 7f2471ffb640 1 -- 10.20.193.121:0/4179735902 wait complete. 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] conn(0x7f972407d4b0 msgr2=0x7f972407f8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] conn(0x7f972407d4b0 0x7f972407f8d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f97240776c0 msgr2=0x7f9724079b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f97240776c0 0x7f9724079b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f974c100970 tx=0x7f9734005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 msgr2=0x7f974c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c19a600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f974000d900 tx=0x7f974000ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 shutdown_connections 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:6808/3393667471,v1:10.20.193.163:6809/3393667471] conn(0x7f972407d4b0 0x7f972407f8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f97240776c0 0x7f9724079b80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f974c1075f0 0x7f974c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 --2- 10.20.193.121:0/810938439 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f974c101540 0x7f974c19a600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 >> 10.20.193.121:0/810938439 conn(0x7f974c0fd220 msgr2=0x7f974c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.192 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 shutdown_connections 2026-02-13T10:39:21.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 --> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6af8063390 con 0x7f6ad407d4b0 2026-02-13T10:39:21.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f9750aba640 1 -- 10.20.193.121:0/810938439 wait complete. 2026-02-13T10:39:21.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6ae67fc640 1 -- 10.20.193.121:0/2423503691 <== osd.5 v2:10.20.193.163:6816/2139375176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6af8063390 con 0x7f6ad407d4b0 2026-02-13T10:39:21.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] conn(0x7f6ad407d4b0 msgr2=0x7f6ad407f8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] conn(0x7f6ad407d4b0 0x7f6ad407f8d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.193 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6ad40776c0 msgr2=0x7f6ad4079b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6ad40776c0 0x7f6ad4079b80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005e90 tx=0x7f6ae803a040 comp rx=0 tx=0).stop 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 msgr2=0x7f6af81a2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af81a2b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00c910 tx=0x7f6aec00cde0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 shutdown_connections 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:6816/2139375176,v1:10.20.193.163:6817/2139375176] conn(0x7f6ad407d4b0 0x7f6ad407f8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6ad40776c0 0x7f6ad4079b80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6af810a8c0 0x7f6af81a2b40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 --2- 10.20.193.121:0/2423503691 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6af81048c0 0x7f6af81a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 >> 10.20.193.121:0/2423503691 conn(0x7f6af81005c0 msgr2=0x7f6af810ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 shutdown_connections 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.151+0000 7f6aff9ab640 1 -- 10.20.193.121:0/2423503691 wait complete. 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 --> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d500637a0 con 0x7f6d2c07d390 2026-02-13T10:39:21.194 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d3e7fc640 1 -- 10.20.193.121:0/2799957656 <== osd.3 v2:10.20.193.163:6800/2365834735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d500637a0 con 0x7f6d2c07d390 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] conn(0x7f6d2c07d390 msgr2=0x7f6d2c07f7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] conn(0x7f6d2c07d390 0x7f6d2c07f7b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6d2c0775a0 msgr2=0x7f6d2c079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6d2c0775a0 0x7f6d2c079a60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6d50102ed0 tx=0x7f6d44008040 comp rx=0 tx=0).stop 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 msgr2=0x7f6d50101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50101440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002f820 tx=0x7f6d40004060 comp rx=0 tx=0).stop 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 shutdown_connections 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:6800/2365834735,v1:10.20.193.163:6801/2365834735] conn(0x7f6d2c07d390 0x7f6d2c07f7b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f6d2c0775a0 0x7f6d2c079a60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f6d50104180 0x7f6d50101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 --2- 10.20.193.121:0/2799957656 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f6d50103860 0x7f6d50101440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 >> 10.20.193.121:0/2799957656 conn(0x7f6d500fd220 msgr2=0x7f6d50104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 shutdown_connections 2026-02-13T10:39:21.195 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.155+0000 7f6d576e7640 1 -- 10.20.193.121:0/2799957656 wait complete. 2026-02-13T10:39:21.197 INFO:teuthology.orchestra.run.trial121.stdout:137438953478 2026-02-13T10:39:21.197 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.4 2026-02-13T10:39:21.199 INFO:teuthology.orchestra.run.trial121.stdout:60129542157 2026-02-13T10:39:21.200 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-13T10:39:21.203 INFO:teuthology.orchestra.run.trial121.stdout:111669149703 2026-02-13T10:39:21.203 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-13T10:39:21.207 INFO:teuthology.orchestra.run.trial121.stdout:77309411339 2026-02-13T10:39:21.207 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-13T10:39:21.224 INFO:teuthology.orchestra.run.trial121.stdout:42949672975 2026-02-13T10:39:21.224 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-13T10:39:21.242 INFO:teuthology.orchestra.run.trial121.stdout:154618822660 2026-02-13T10:39:21.243 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.5 2026-02-13T10:39:21.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:20 trial163 ceph-mon[24184]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:21.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:20 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2695538318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2026-02-13T10:39:21.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:20 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/421424610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2026-02-13T10:39:21.511 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.535 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.536 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.537 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.630 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/3762663544 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac104180 msgr2=0x7fb1ac10c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/3762663544 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac104180 0x7fb1ac10c670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009910 tx=0x7fb1a002f260 comp rx=0 tx=0).stop 2026-02-13T10:39:21.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/3762663544 shutdown_connections 2026-02-13T10:39:21.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/3762663544 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac104180 0x7fb1ac10c670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/3762663544 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb1ac103860 0x7fb1ac103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/3762663544 >> 10.20.193.121:0/3762663544 conn(0x7fb1ac0fd220 msgr2=0x7fb1ac0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.633 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/3762663544 shutdown_connections 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/3762663544 wait complete. 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 Processor -- start 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- start start 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac103860 0x7fb1ac1a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb1ac104180 0x7fb1ac1a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10fa00 con 0x7fb1ac103860 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b3a2e640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac10f880 con 0x7fb1ac104180 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b17a3640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac103860 0x7fb1ac1a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b17a3640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac103860 0x7fb1ac1a2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50432/0 (socket says 10.20.193.121:50432) 2026-02-13T10:39:21.634 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.630+0000 7fb1b17a3640 1 -- 10.20.193.121:0/303967182 learned_addr learned my addr 10.20.193.121:0/303967182 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.635 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b0fa2640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb1ac104180 0x7fb1ac1a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.635 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b17a3640 1 -- 10.20.193.121:0/303967182 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb1ac104180 msgr2=0x7fb1ac1a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.635 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b17a3640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb1ac104180 0x7fb1ac1a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.635 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b17a3640 1 -- 10.20.193.121:0/303967182 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac19c7b0 con 0x7fb1ac103860 2026-02-13T10:39:21.635 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:21.637 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b17a3640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac103860 0x7fb1ac1a2630 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e9b0 tx=0x7fb19c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.637 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb19a7fc640 1 -- 10.20.193.121:0/303967182 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fb19c00cbe0 con 0x7fb1ac103860 2026-02-13T10:39:21.637 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac19caa0 con 0x7fb1ac103860 2026-02-13T10:39:21.637 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb19a7fc640 1 -- 10.20.193.121:0/303967182 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fb19c004560 con 0x7fb1ac103860 2026-02-13T10:39:21.637 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb19a7fc640 1 -- 10.20.193.121:0/303967182 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fb19c010430 con 0x7fb1ac103860 2026-02-13T10:39:21.637 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac19d030 con 0x7fb1ac103860 2026-02-13T10:39:21.637 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.631+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac06b6d0 con 0x7fb1ac103860 2026-02-13T10:39:21.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.632+0000 7fb19a7fc640 1 -- 10.20.193.121:0/303967182 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fb19c0040d0 con 0x7fb1ac103860 2026-02-13T10:39:21.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.633+0000 7fb19a7fc640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb1880775a0 0x7fb188079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.633+0000 7fb19a7fc640 1 -- 10.20.193.121:0/303967182 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7fb19c0998d0 con 0x7fb1ac103860 2026-02-13T10:39:21.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.633+0000 7fb1b0fa2640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb1880775a0 0x7fb188079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.633+0000 7fb1b0fa2640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb1880775a0 0x7fb188079a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac10bae0 tx=0x7fb1a0005d90 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.635+0000 7fb19a7fc640 1 -- 10.20.193.121:0/303967182 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb19c066d40 con 0x7fb1ac103860 2026-02-13T10:39:21.641 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 -- 10.20.193.121:0/720576947 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f588410a8c0 msgr2=0x7f588410aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.641 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 --2- 10.20.193.121:0/720576947 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f588410a8c0 0x7f588410aca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f58780099b0 tx=0x7f587802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.641 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 -- 10.20.193.121:0/720576947 shutdown_connections 2026-02-13T10:39:21.641 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 --2- 10.20.193.121:0/720576947 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f58841048c0 0x7f5884104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.641 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 --2- 10.20.193.121:0/720576947 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f588410a8c0 0x7f588410aca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.641 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 -- 10.20.193.121:0/720576947 >> 10.20.193.121:0/720576947 conn(0x7f58841005c0 msgr2=0x7f58841029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.641 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 -- 10.20.193.121:0/720576947 shutdown_connections 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.636+0000 7f5889905640 1 -- 10.20.193.121:0/720576947 wait complete. 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 Processor -- start 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 -- start start 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f58841048c0 0x7f58841a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f588410a8c0 0x7f58841a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5884111ca0 con 0x7f58841048c0 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5884111b20 con 0x7f588410a8c0 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5882ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f58841048c0 0x7f58841a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5882ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f58841048c0 0x7f58841a26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50444/0 (socket says 10.20.193.121:50444) 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5882ffd640 1 -- 10.20.193.121:0/1713226746 learned_addr learned my addr 10.20.193.121:0/1713226746 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5882ffd640 1 -- 10.20.193.121:0/1713226746 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f588410a8c0 msgr2=0x7f58841a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.642 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5882ffd640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f588410a8c0 0x7f58841a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5882ffd640 1 -- 10.20.193.121:0/1713226746 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588419c890 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5882ffd640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f58841048c0 0x7f58841a26d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5878009980 tx=0x7f5878002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5888903640 1 -- 10.20.193.121:0/1713226746 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5878004420 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5888903640 1 -- 10.20.193.121:0/1713226746 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f5878038c00 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588419cb20 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5888903640 1 -- 10.20.193.121:0/1713226746 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5878041ab0 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.637+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588419d050 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.638+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5884106000 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.638+0000 7f5888903640 1 -- 10.20.193.121:0/1713226746 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f5878038770 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.639+0000 7f5888903640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5858077550 0x7f5858079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.639+0000 7f5888903640 1 -- 10.20.193.121:0/1713226746 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f587803e070 con 0x7f58841048c0 2026-02-13T10:39:21.643 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.639+0000 7f58827fc640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5858077550 0x7f5858079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.644 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.639+0000 7f58827fc640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5858077550 0x7f5858079a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f588410a0b0 tx=0x7f586c006d90 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.644 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.642+0000 7f5888903640 1 -- 10.20.193.121:0/1713226746 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f587808b790 con 0x7f58841048c0 2026-02-13T10:39:21.646 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 -- 10.20.193.121:0/513710796 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 msgr2=0x7fcd44102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.646 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 --2- 10.20.193.121:0/513710796 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44102ac0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0099b0 tx=0x7fcd2c02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 -- 10.20.193.121:0/513710796 shutdown_connections 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 --2- 10.20.193.121:0/513710796 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44102ac0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 --2- 10.20.193.121:0/513710796 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcd44108660 0x7fcd44108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 -- 10.20.193.121:0/513710796 >> 10.20.193.121:0/513710796 conn(0x7fcd440fe380 msgr2=0x7fcd441007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 -- 10.20.193.121:0/513710796 shutdown_connections 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 -- 10.20.193.121:0/513710796 wait complete. 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 Processor -- start 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.643+0000 7fcd49a67640 1 -- start start 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd49a67640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.647 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd49a67640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcd44108660 0x7fcd440758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd49a67640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4410f9c0 con 0x7fcd44102660 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd49a67640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4410f840 con 0x7fcd44108660 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd42ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd42ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50454/0 (socket says 10.20.193.121:50454) 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd42ffd640 1 -- 10.20.193.121:0/1659171668 learned_addr learned my addr 10.20.193.121:0/1659171668 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd427fc640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcd44108660 0x7fcd440758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd42ffd640 1 -- 10.20.193.121:0/1659171668 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcd44108660 msgr2=0x7fcd440758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd42ffd640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcd44108660 0x7fcd440758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd42ffd640 1 -- 10.20.193.121:0/1659171668 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd44075df0 con 0x7fcd44102660 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd427fc640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcd44108660 0x7fcd440758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd42ffd640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44075370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800e990 tx=0x7fcd3800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.648 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd48a65640 1 -- 10.20.193.121:0/1659171668 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fcd3800cb80 con 0x7fcd44102660 2026-02-13T10:39:21.649 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd440760e0 con 0x7fcd44102660 2026-02-13T10:39:21.649 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd48a65640 1 -- 10.20.193.121:0/1659171668 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fcd38004560 con 0x7fcd44102660 2026-02-13T10:39:21.649 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd48a65640 1 -- 10.20.193.121:0/1659171668 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fcd38010430 con 0x7fcd44102660 2026-02-13T10:39:21.649 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.644+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd441a6bf0 con 0x7fcd44102660 2026-02-13T10:39:21.649 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.645+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd44103da0 con 0x7fcd44102660 2026-02-13T10:39:21.650 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.645+0000 7fcd48a65640 1 -- 10.20.193.121:0/1659171668 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fcd38020030 con 0x7fcd44102660 2026-02-13T10:39:21.650 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.646+0000 7fcd48a65640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcd18077550 0x7fcd18079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.650 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.646+0000 7fcd427fc640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcd18077550 0x7fcd18079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.650 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.646+0000 7fcd48a65640 1 -- 10.20.193.121:0/1659171668 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7fcd380997d0 con 0x7fcd44102660 2026-02-13T10:39:21.650 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.646+0000 7fcd427fc640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcd18077550 0x7fcd18079a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0097c0 tx=0x7fcd2c0384d0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.651 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.649+0000 7fcd48a65640 1 -- 10.20.193.121:0/1659171668 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcd3801f2f0 con 0x7fcd44102660 2026-02-13T10:39:21.660 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 -- 10.20.193.121:0/1635449911 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c41048c0 msgr2=0x7f39c4104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/1635449911 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c41048c0 0x7f39c4104d20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39b40099b0 tx=0x7f39b402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 -- 10.20.193.121:0/1635449911 shutdown_connections 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/1635449911 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c41048c0 0x7f39c4104d20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/1635449911 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f39c410a8c0 0x7f39c410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 -- 10.20.193.121:0/1635449911 >> 10.20.193.121:0/1635449911 conn(0x7f39c41005c0 msgr2=0x7f39c41029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 -- 10.20.193.121:0/1635449911 shutdown_connections 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.657+0000 7f39ca9ee640 1 -- 10.20.193.121:0/1635449911 wait complete. 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39ca9ee640 1 Processor -- start 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39ca9ee640 1 -- start start 2026-02-13T10:39:21.661 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39ca9ee640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f39c41048c0 0x7f39c41a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39ca9ee640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c410a8c0 0x7f39c41a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39ca9ee640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4111c20 con 0x7f39c410a8c0 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39ca9ee640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4111aa0 con 0x7f39c41048c0 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39c37fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c410a8c0 0x7f39c41a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39c37fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c410a8c0 0x7f39c41a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50474/0 (socket says 10.20.193.121:50474) 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39c37fe640 1 -- 10.20.193.121:0/3009400880 learned_addr learned my addr 10.20.193.121:0/3009400880 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39c3fff640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f39c41048c0 0x7f39c41a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39c37fe640 1 -- 10.20.193.121:0/3009400880 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f39c41048c0 msgr2=0x7f39c41a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39c37fe640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f39c41048c0 0x7f39c41a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.658+0000 7f39c37fe640 1 -- 10.20.193.121:0/3009400880 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c419c7b0 con 0x7f39c410a8c0 2026-02-13T10:39:21.662 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.659+0000 7f39c37fe640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c410a8c0 0x7f39c41a2b70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39b4002410 tx=0x7f39b40028f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.663 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.659+0000 7f39c17fa640 1 -- 10.20.193.121:0/3009400880 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f39b4004210 con 0x7f39c410a8c0 2026-02-13T10:39:21.663 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.659+0000 7f39c17fa640 1 -- 10.20.193.121:0/3009400880 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f39b40043b0 con 0x7f39c410a8c0 2026-02-13T10:39:21.663 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.659+0000 7f39c17fa640 1 -- 10.20.193.121:0/3009400880 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f39b4041820 con 0x7f39c410a8c0 2026-02-13T10:39:21.663 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.659+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c419ca40 con 0x7f39c410a8c0 2026-02-13T10:39:21.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.659+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c419cf70 con 0x7f39c410a8c0 2026-02-13T10:39:21.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.660+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c4106000 con 0x7f39c410a8c0 2026-02-13T10:39:21.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.660+0000 7f39c17fa640 1 -- 10.20.193.121:0/3009400880 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f39b40419c0 con 0x7f39c410a8c0 2026-02-13T10:39:21.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.661+0000 7f39c17fa640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f39980775a0 0x7f3998079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.661+0000 7f39c3fff640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f39980775a0 0x7f3998079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.661+0000 7f39c17fa640 1 -- 10.20.193.121:0/3009400880 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f39b403e070 con 0x7f39c410a8c0 2026-02-13T10:39:21.664 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.661+0000 7f39c3fff640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f39980775a0 0x7f3998079a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39b0002930 tx=0x7f39b0005f50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.665 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.663+0000 7f39c17fa640 1 -- 10.20.193.121:0/3009400880 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f39b4004560 con 0x7f39c410a8c0 2026-02-13T10:39:21.725 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.724+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb1ac19d2e0 con 0x7fb1ac103860 2026-02-13T10:39:21.726 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.724+0000 7fb19a7fc640 1 -- 10.20.193.121:0/303967182 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb19c06bbf0 con 0x7fb1ac103860 2026-02-13T10:39:21.726 INFO:teuthology.orchestra.run.trial121.stdout:137438953476 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb1880775a0 msgr2=0x7fb188079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb1880775a0 0x7fb188079a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac10bae0 tx=0x7fb1a0005d90 comp rx=0 tx=0).stop 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac103860 msgr2=0x7fb1ac1a2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac103860 0x7fb1ac1a2630 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e9b0 tx=0x7fb19c00ee80 comp rx=0 tx=0).stop 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 shutdown_connections 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fb1880775a0 0x7fb188079a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fb1ac104180 0x7fb1ac1a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 --2- 10.20.193.121:0/303967182 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fb1ac103860 0x7fb1ac1a2630 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 >> 10.20.193.121:0/303967182 conn(0x7fb1ac0fd220 msgr2=0x7fb1ac10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.729 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 shutdown_connections 2026-02-13T10:39:21.729 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.726+0000 7fb1b3a2e640 1 -- 10.20.193.121:0/303967182 wait complete. 2026-02-13T10:39:21.734 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.733+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f588419de90 con 0x7f58841048c0 2026-02-13T10:39:21.735 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.733+0000 7f5888903640 1 -- 10.20.193.121:0/1713226746 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5878090640 con 0x7f58841048c0 2026-02-13T10:39:21.735 INFO:teuthology.orchestra.run.trial121.stdout:60129542156 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5858077550 msgr2=0x7f5858079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5858077550 0x7f5858079a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f588410a0b0 tx=0x7f586c006d90 comp rx=0 tx=0).stop 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f58841048c0 msgr2=0x7f58841a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f58841048c0 0x7f58841a26d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5878009980 tx=0x7f5878002e20 comp rx=0 tx=0).stop 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 shutdown_connections 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5858077550 0x7f5858079a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f588410a8c0 0x7f58841a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 --2- 10.20.193.121:0/1713226746 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f58841048c0 0x7f58841a26d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 >> 10.20.193.121:0/1713226746 conn(0x7f58841005c0 msgr2=0x7f58841016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 shutdown_connections 2026-02-13T10:39:21.737 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.735+0000 7f5889905640 1 -- 10.20.193.121:0/1713226746 wait complete. 2026-02-13T10:39:21.740 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.738+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcd4410a700 con 0x7fcd44102660 2026-02-13T10:39:21.740 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.738+0000 7fcd48a65640 1 -- 10.20.193.121:0/1659171668 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcd38066bc0 con 0x7fcd44102660 2026-02-13T10:39:21.740 INFO:teuthology.orchestra.run.trial121.stdout:77309411338 2026-02-13T10:39:21.742 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcd18077550 msgr2=0x7fcd18079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.742 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcd18077550 0x7fcd18079a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0097c0 tx=0x7fcd2c0384d0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.742 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 msgr2=0x7fcd44075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.742 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44075370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800e990 tx=0x7fcd3800ee60 comp rx=0 tx=0).stop 2026-02-13T10:39:21.743 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 shutdown_connections 2026-02-13T10:39:21.743 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fcd18077550 0x7fcd18079a10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.743 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fcd44108660 0x7fcd440758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.743 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 --2- 10.20.193.121:0/1659171668 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fcd44102660 0x7fcd44075370 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.743 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 >> 10.20.193.121:0/1659171668 conn(0x7fcd440fe380 msgr2=0x7fcd440fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.743 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 shutdown_connections 2026-02-13T10:39:21.743 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.740+0000 7fcd49a67640 1 -- 10.20.193.121:0/1659171668 wait complete. 2026-02-13T10:39:21.753 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 -- 10.20.193.121:0/3908546409 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f370010a8c0 msgr2=0x7f370010aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.753 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 --2- 10.20.193.121:0/3908546409 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f370010a8c0 0x7f370010aca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f36f00099b0 tx=0x7f36f002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:21.753 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 -- 10.20.193.121:0/3908546409 shutdown_connections 2026-02-13T10:39:21.753 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 --2- 10.20.193.121:0/3908546409 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f37001048c0 0x7f3700104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.753 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 --2- 10.20.193.121:0/3908546409 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f370010a8c0 0x7f370010aca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.754 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 -- 10.20.193.121:0/3908546409 >> 10.20.193.121:0/3908546409 conn(0x7f37001005c0 msgr2=0x7f37001029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.754 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 -- 10.20.193.121:0/3908546409 shutdown_connections 2026-02-13T10:39:21.754 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f370804b640 1 -- 10.20.193.121:0/3908546409 wait complete. 2026-02-13T10:39:21.754 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 -- 10.20.193.121:0/2093431785 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300104180 msgr2=0x7f330010c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 --2- 10.20.193.121:0/2093431785 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300104180 0x7f330010c670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0099e0 tx=0x7f32ec02f2f0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 -- 10.20.193.121:0/2093431785 shutdown_connections 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 --2- 10.20.193.121:0/2093431785 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300104180 0x7f330010c670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 --2- 10.20.193.121:0/2093431785 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3300103860 0x7f3300103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 -- 10.20.193.121:0/2093431785 >> 10.20.193.121:0/2093431785 conn(0x7f33000fd220 msgr2=0x7f33000ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 -- 10.20.193.121:0/2093431785 shutdown_connections 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.751+0000 7f3305c72640 1 -- 10.20.193.121:0/2093431785 wait complete. 2026-02-13T10:39:21.755 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3305c72640 1 Processor -- start 2026-02-13T10:39:21.756 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f370804b640 1 Processor -- start 2026-02-13T10:39:21.756 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3305c72640 1 -- start start 2026-02-13T10:39:21.756 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f370804b640 1 -- start start 2026-02-13T10:39:21.757 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3305c72640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300103860 0x7f3300101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.757 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3305c72640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3300104180 0x7f3300101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.757 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3305c72640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f330010fa00 con 0x7f3300103860 2026-02-13T10:39:21.757 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3305c72640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f330010f880 con 0x7f3300104180 2026-02-13T10:39:21.758 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f370804b640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f37001048c0 0x7f37001a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.758 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f370804b640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f370010a8c0 0x7f37001a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.758 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f370804b640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3700111c40 con 0x7f37001048c0 2026-02-13T10:39:21.758 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f370804b640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3700111ac0 con 0x7f370010a8c0 2026-02-13T10:39:21.758 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32ff7fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300103860 0x7f3300101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32ff7fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300103860 0x7f3300101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50514/0 (socket says 10.20.193.121:50514) 2026-02-13T10:39:21.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32ff7fe640 1 -- 10.20.193.121:0/282249283 learned_addr learned my addr 10.20.193.121:0/282249283 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3705dc0640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f37001048c0 0x7f37001a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3705dc0640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f37001048c0 0x7f37001a2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50516/0 (socket says 10.20.193.121:50516) 2026-02-13T10:39:21.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3705dc0640 1 -- 10.20.193.121:0/2916382789 learned_addr learned my addr 10.20.193.121:0/2916382789 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:21.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32feffd640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3300104180 0x7f3300101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f37055bf640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f370010a8c0 0x7f37001a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32ff7fe640 1 -- 10.20.193.121:0/282249283 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3300104180 msgr2=0x7f3300101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32ff7fe640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3300104180 0x7f3300101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32ff7fe640 1 -- 10.20.193.121:0/282249283 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3300101ec0 con 0x7f3300103860 2026-02-13T10:39:21.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32feffd640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3300104180 0x7f3300101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:21.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32ff7fe640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300103860 0x7f3300101440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d900 tx=0x7f32f000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3705dc0640 1 -- 10.20.193.121:0/2916382789 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f370010a8c0 msgr2=0x7f37001a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3705dc0640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f370010a8c0 0x7f37001a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3705dc0640 1 -- 10.20.193.121:0/2916382789 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370019c7b0 con 0x7f37001048c0 2026-02-13T10:39:21.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32fcff9640 1 -- 10.20.193.121:0/282249283 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f32f0014070 con 0x7f3300103860 2026-02-13T10:39:21.763 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33001021b0 con 0x7f3300103860 2026-02-13T10:39:21.763 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32fcff9640 1 -- 10.20.193.121:0/282249283 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f32f00044a0 con 0x7f3300103860 2026-02-13T10:39:21.763 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f32fcff9640 1 -- 10.20.193.121:0/282249283 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f32f0005020 con 0x7f3300103860 2026-02-13T10:39:21.763 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.752+0000 7f3705dc0640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f37001048c0 0x7f37001a2630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36f0002410 tx=0x7f36f00028f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.763 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.753+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33001a6c70 con 0x7f3300103860 2026-02-13T10:39:21.764 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.753+0000 7f36eeffd640 1 -- 10.20.193.121:0/2916382789 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f36f0004210 con 0x7f37001048c0 2026-02-13T10:39:21.764 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.753+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370019ca40 con 0x7f37001048c0 2026-02-13T10:39:21.764 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.753+0000 7f36eeffd640 1 -- 10.20.193.121:0/2916382789 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f36f00043b0 con 0x7f37001048c0 2026-02-13T10:39:21.764 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.753+0000 7f36eeffd640 1 -- 10.20.193.121:0/2916382789 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f36f0041820 con 0x7f37001048c0 2026-02-13T10:39:21.764 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.753+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370019cf70 con 0x7f37001048c0 2026-02-13T10:39:21.765 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.753+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330006b6d0 con 0x7f3300103860 2026-02-13T10:39:21.765 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.754+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3700106000 con 0x7f37001048c0 2026-02-13T10:39:21.765 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.754+0000 7f32fcff9640 1 -- 10.20.193.121:0/282249283 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f32f00052e0 con 0x7f3300103860 2026-02-13T10:39:21.766 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.754+0000 7f36eeffd640 1 -- 10.20.193.121:0/2916382789 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f36f0041ad0 con 0x7f37001048c0 2026-02-13T10:39:21.766 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.754+0000 7f32fcff9640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f32d4077550 0x7f32d4079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.766 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.754+0000 7f32fcff9640 1 -- 10.20.193.121:0/282249283 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f32f0099850 con 0x7f3300103860 2026-02-13T10:39:21.767 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.754+0000 7f32feffd640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f32d4077550 0x7f32d4079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.767 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.755+0000 7f36eeffd640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f36dc077550 0x7f36dc079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:21.767 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.755+0000 7f37055bf640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f36dc077550 0x7f36dc079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:21.767 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.755+0000 7f36eeffd640 1 -- 10.20.193.121:0/2916382789 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f36f003e070 con 0x7f37001048c0 2026-02-13T10:39:21.768 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.755+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f39c419d2b0 con 0x7f39c410a8c0 2026-02-13T10:39:21.768 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.755+0000 7f37055bf640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f36dc077550 0x7f36dc079a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36f4005fd0 tx=0x7f36f4005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.768 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.755+0000 7f32feffd640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f32d4077550 0x7f32d4079a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32ec004870 tx=0x7f32ec004800 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:21.768 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.757+0000 7f32fcff9640 1 -- 10.20.193.121:0/282249283 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f32f0066c40 con 0x7f3300103860 2026-02-13T10:39:21.769 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.759+0000 7f39c17fa640 1 -- 10.20.193.121:0/3009400880 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f39b408b4d0 con 0x7f39c410a8c0 2026-02-13T10:39:21.769 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.759+0000 7f36eeffd640 1 -- 10.20.193.121:0/2916382789 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f36f0004560 con 0x7f37001048c0 2026-02-13T10:39:21.769 INFO:teuthology.orchestra.run.trial121.stdout:111669149702 2026-02-13T10:39:21.770 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.761+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f39980775a0 msgr2=0x7f3998079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.770 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.761+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f39980775a0 0x7f3998079a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39b0002930 tx=0x7f39b0005f50 comp rx=0 tx=0).stop 2026-02-13T10:39:21.770 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.761+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c410a8c0 msgr2=0x7f39c41a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.770 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.761+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c410a8c0 0x7f39c41a2b70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39b4002410 tx=0x7f39b40028f0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.770 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.762+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 shutdown_connections 2026-02-13T10:39:21.770 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.762+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f39980775a0 0x7f3998079a60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.771 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.762+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f39c410a8c0 0x7f39c41a2b70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.771 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.762+0000 7f39ca9ee640 1 --2- 10.20.193.121:0/3009400880 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f39c41048c0 0x7f39c41a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.771 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.762+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 >> 10.20.193.121:0/3009400880 conn(0x7f39c41005c0 msgr2=0x7f39c41016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.771 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.762+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 shutdown_connections 2026-02-13T10:39:21.771 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.762+0000 7f39ca9ee640 1 -- 10.20.193.121:0/3009400880 wait complete. 2026-02-13T10:39:21.772 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953478 got 137438953476 for osd.4 2026-02-13T10:39:21.783 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542157 got 60129542156 for osd.1 2026-02-13T10:39:21.789 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411339 got 77309411338 for osd.2 2026-02-13T10:39:21.804 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149703 got 111669149702 for osd.3 2026-02-13T10:39:21.847 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.845+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3300106980 con 0x7f3300103860 2026-02-13T10:39:21.847 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.845+0000 7f32fcff9640 1 -- 10.20.193.121:0/282249283 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f32f006baf0 con 0x7f3300103860 2026-02-13T10:39:21.847 INFO:teuthology.orchestra.run.trial121.stdout:154618822659 2026-02-13T10:39:21.849 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f32d4077550 msgr2=0x7f32d4079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.849 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f32d4077550 0x7f32d4079a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32ec004870 tx=0x7f32ec004800 comp rx=0 tx=0).stop 2026-02-13T10:39:21.849 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300103860 msgr2=0x7f3300101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.849 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300103860 0x7f3300101440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d900 tx=0x7f32f000ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.849 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 shutdown_connections 2026-02-13T10:39:21.850 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f32d4077550 0x7f32d4079a10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.850 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f3300104180 0x7f3300101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.850 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 --2- 10.20.193.121:0/282249283 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f3300103860 0x7f3300101440 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.850 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 >> 10.20.193.121:0/282249283 conn(0x7f33000fd220 msgr2=0x7f3300104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.850 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 shutdown_connections 2026-02-13T10:39:21.850 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.847+0000 7f3305c72640 1 -- 10.20.193.121:0/282249283 wait complete. 2026-02-13T10:39:21.854 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.853+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f370019d220 con 0x7f37001048c0 2026-02-13T10:39:21.854 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.853+0000 7f36eeffd640 1 -- 10.20.193.121:0/2916382789 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36f008b4f0 con 0x7f37001048c0 2026-02-13T10:39:21.855 INFO:teuthology.orchestra.run.trial121.stdout:42949672974 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f36dc077550 msgr2=0x7f36dc079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f36dc077550 0x7f36dc079a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36f4005fd0 tx=0x7f36f4005950 comp rx=0 tx=0).stop 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f37001048c0 msgr2=0x7f37001a2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f37001048c0 0x7f37001a2630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36f0002410 tx=0x7f36f00028f0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 shutdown_connections 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f36dc077550 0x7f36dc079a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f370010a8c0 0x7f37001a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.857 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 --2- 10.20.193.121:0/2916382789 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f37001048c0 0x7f37001a2630 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:21.858 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 >> 10.20.193.121:0/2916382789 conn(0x7f37001005c0 msgr2=0x7f37001016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:21.858 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 shutdown_connections 2026-02-13T10:39:21.858 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:21.855+0000 7f370804b640 1 -- 10.20.193.121:0/2916382789 wait complete. 2026-02-13T10:39:21.887 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822660 got 154618822659 for osd.5 2026-02-13T10:39:21.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 42949672975 got 42949672974 for osd.0 2026-02-13T10:39:22.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:21 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/303967182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2026-02-13T10:39:22.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:21 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1713226746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2026-02-13T10:39:22.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:21 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1659171668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2026-02-13T10:39:22.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:21 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3009400880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2026-02-13T10:39:22.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:21 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/282249283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2026-02-13T10:39:22.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:21 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2916382789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2026-02-13T10:39:22.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:21 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/303967182' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2026-02-13T10:39:22.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:21 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1713226746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2026-02-13T10:39:22.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:21 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1659171668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2026-02-13T10:39:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:21 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3009400880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2026-02-13T10:39:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:21 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/282249283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2026-02-13T10:39:22.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:21 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2916382789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2026-02-13T10:39:22.773 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.4 2026-02-13T10:39:22.784 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-13T10:39:22.789 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-13T10:39:22.805 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-13T10:39:22.888 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.5 2026-02-13T10:39:22.899 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-13T10:39:23.090 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:23.124 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:22 trial121 ceph-mon[19159]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:23.139 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:23.139 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:23.187 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:23.219 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:23.234 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 -- 10.20.193.121:0/781111435 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 msgr2=0x7efc20102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 --2- 10.20.193.121:0/781111435 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc20102ac0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efc080099b0 tx=0x7efc0802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 -- 10.20.193.121:0/781111435 shutdown_connections 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 --2- 10.20.193.121:0/781111435 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc20102ac0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 --2- 10.20.193.121:0/781111435 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7efc20108660 0x7efc20108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 -- 10.20.193.121:0/781111435 >> 10.20.193.121:0/781111435 conn(0x7efc200fe380 msgr2=0x7efc201007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 -- 10.20.193.121:0/781111435 shutdown_connections 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 -- 10.20.193.121:0/781111435 wait complete. 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.231+0000 7efc2541c640 1 Processor -- start 2026-02-13T10:39:23.235 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc2541c640 1 -- start start 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc2541c640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc2019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc2541c640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7efc20108660 0x7efc2019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc2541c640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc2010f9c0 con 0x7efc20102660 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc2541c640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7efc2010f840 con 0x7efc20108660 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1effd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc2019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1effd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc2019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50544/0 (socket says 10.20.193.121:50544) 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1effd640 1 -- 10.20.193.121:0/3726685448 learned_addr learned my addr 10.20.193.121:0/3726685448 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1e7fc640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7efc20108660 0x7efc2019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1effd640 1 -- 10.20.193.121:0/3726685448 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7efc20108660 msgr2=0x7efc2019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1effd640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7efc20108660 0x7efc2019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.236 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1effd640 1 -- 10.20.193.121:0/3726685448 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2019ef70 con 0x7efc20102660 2026-02-13T10:39:23.237 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc1effd640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc2019a6a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc1400d8d0 tx=0x7efc1400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.237 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efbfffff640 1 -- 10.20.193.121:0/3726685448 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7efc14014070 con 0x7efc20102660 2026-02-13T10:39:23.237 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2019f260 con 0x7efc20102660 2026-02-13T10:39:23.237 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efbfffff640 1 -- 10.20.193.121:0/3726685448 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7efc140044a0 con 0x7efc20102660 2026-02-13T10:39:23.237 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efbfffff640 1 -- 10.20.193.121:0/3726685448 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7efc14002e60 con 0x7efc20102660 2026-02-13T10:39:23.237 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.232+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2019f7f0 con 0x7efc20102660 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- 10.20.193.121:0/3260491549 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff5381048c0 msgr2=0x7ff538104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/3260491549 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff5381048c0 0x7ff538104d20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff5200099b0 tx=0x7ff52002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- 10.20.193.121:0/3260491549 shutdown_connections 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/3260491549 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff5381048c0 0x7ff538104d20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/3260491549 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff53810a8c0 0x7ff53810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- 10.20.193.121:0/3260491549 >> 10.20.193.121:0/3260491549 conn(0x7ff5381005c0 msgr2=0x7ff5381029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- 10.20.193.121:0/3260491549 shutdown_connections 2026-02-13T10:39:23.238 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- 10.20.193.121:0/3260491549 wait complete. 2026-02-13T10:39:23.239 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc20103da0 con 0x7efc20102660 2026-02-13T10:39:23.240 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 Processor -- start 2026-02-13T10:39:23.240 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- start start 2026-02-13T10:39:23.240 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff5381048c0 0x7ff5381a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.240 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff53810a8c0 0x7ff5381a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111c20 con 0x7ff53810a8c0 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff53ee6f640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff538111aa0 con 0x7ff5381048c0 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff52ffff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff53810a8c0 0x7ff5381a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff52ffff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff53810a8c0 0x7ff5381a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50560/0 (socket says 10.20.193.121:50560) 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7ff52ffff640 1 -- 10.20.193.121:0/4000109276 learned_addr learned my addr 10.20.193.121:0/4000109276 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7ff53cbe4640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff5381048c0 0x7ff5381a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7ff52ffff640 1 -- 10.20.193.121:0/4000109276 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff5381048c0 msgr2=0x7ff5381a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7ff52ffff640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff5381048c0 0x7ff5381a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.241 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7ff52ffff640 1 -- 10.20.193.121:0/4000109276 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53819c7b0 con 0x7ff53810a8c0 2026-02-13T10:39:23.242 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.233+0000 7efbfffff640 1 -- 10.20.193.121:0/3726685448 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7efc1400b840 con 0x7efc20102660 2026-02-13T10:39:23.242 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7ff52ffff640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff53810a8c0 0x7ff5381a2b70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff520002410 tx=0x7ff5200028f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.243 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7efbfffff640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7efbf0077540 0x7efbf0079a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.243 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7efc1e7fc640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7efbf0077540 0x7efbf0079a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.243 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7efbfffff640 1 -- 10.20.193.121:0/3726685448 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7efc14099700 con 0x7efc20102660 2026-02-13T10:39:23.243 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.234+0000 7efc1e7fc640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7efbf0077540 0x7efbf0079a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc20100b40 tx=0x7efc0803a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.243 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.236+0000 7ff52dffb640 1 -- 10.20.193.121:0/4000109276 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ff520004210 con 0x7ff53810a8c0 2026-02-13T10:39:23.244 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.236+0000 7ff52dffb640 1 -- 10.20.193.121:0/4000109276 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7ff5200043b0 con 0x7ff53810a8c0 2026-02-13T10:39:23.244 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.236+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53819ca40 con 0x7ff53810a8c0 2026-02-13T10:39:23.244 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.236+0000 7ff52dffb640 1 -- 10.20.193.121:0/4000109276 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7ff520041820 con 0x7ff53810a8c0 2026-02-13T10:39:23.244 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.236+0000 7efbfffff640 1 -- 10.20.193.121:0/3726685448 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efc14066b10 con 0x7efc20102660 2026-02-13T10:39:23.244 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53819cf70 con 0x7ff53810a8c0 2026-02-13T10:39:23.245 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- 10.20.193.121:0/1495306668 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077620 msgr2=0x7f7400077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.245 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 --2- 10.20.193.121:0/1495306668 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077620 0x7f7400077a00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009a00 tx=0x7f73f002f280 comp rx=0 tx=0).stop 2026-02-13T10:39:23.245 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- 10.20.193.121:0/1495306668 shutdown_connections 2026-02-13T10:39:23.245 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 --2- 10.20.193.121:0/1495306668 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7400077f40 0x7f7400113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.245 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 --2- 10.20.193.121:0/1495306668 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077620 0x7f7400077a00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.246 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- 10.20.193.121:0/1495306668 >> 10.20.193.121:0/1495306668 conn(0x7f74001005c0 msgr2=0x7f74001029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.246 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- 10.20.193.121:0/1495306668 shutdown_connections 2026-02-13T10:39:23.246 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- 10.20.193.121:0/1495306668 wait complete. 2026-02-13T10:39:23.246 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff538106000 con 0x7ff53810a8c0 2026-02-13T10:39:23.247 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 Processor -- start 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- start start 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7400077620 0x7f74001a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077f40 0x7f74001a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7400116160 con 0x7f7400077f40 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.237+0000 7f7407e19640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7400115fe0 con 0x7f7400077620 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f740538d640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077f40 0x7f74001a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f740538d640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077f40 0x7f74001a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50578/0 (socket says 10.20.193.121:50578) 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f740538d640 1 -- 10.20.193.121:0/2407092200 learned_addr learned my addr 10.20.193.121:0/2407092200 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f7405b8e640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7400077620 0x7f74001a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.248 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f740538d640 1 -- 10.20.193.121:0/2407092200 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7400077620 msgr2=0x7f74001a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.249 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f740538d640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7400077620 0x7f74001a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.249 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f740538d640 1 -- 10.20.193.121:0/2407092200 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74001a56b0 con 0x7f7400077f40 2026-02-13T10:39:23.249 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7f740538d640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077f40 0x7f74001a1320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f73f400d8d0 tx=0x7f73f400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.249 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7ff52dffb640 1 -- 10.20.193.121:0/4000109276 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7ff520041ad0 con 0x7ff53810a8c0 2026-02-13T10:39:23.250 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7ff52dffb640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff5140775a0 0x7ff514079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.250 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7ff52dffb640 1 -- 10.20.193.121:0/4000109276 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7ff52003e070 con 0x7ff53810a8c0 2026-02-13T10:39:23.250 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.238+0000 7ff53cbe4640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff5140775a0 0x7ff514079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.250 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.239+0000 7ff53cbe4640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff5140775a0 0x7ff514079a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff528002930 tx=0x7ff528005f50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.250 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.241+0000 7f73eeffd640 1 -- 10.20.193.121:0/2407092200 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f73f4014070 con 0x7f7400077f40 2026-02-13T10:39:23.250 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.241+0000 7f73eeffd640 1 -- 10.20.193.121:0/2407092200 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f73f40044a0 con 0x7f7400077f40 2026-02-13T10:39:23.251 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.241+0000 7ff52dffb640 1 -- 10.20.193.121:0/4000109276 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff520004560 con 0x7ff53810a8c0 2026-02-13T10:39:23.252 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.241+0000 7f73eeffd640 1 -- 10.20.193.121:0/2407092200 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f73f4002e60 con 0x7f7400077f40 2026-02-13T10:39:23.252 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.241+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74001a59a0 con 0x7f7400077f40 2026-02-13T10:39:23.252 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.241+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001a5f00 con 0x7f7400077f40 2026-02-13T10:39:23.252 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.242+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7400078d60 con 0x7f7400077f40 2026-02-13T10:39:23.252 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.242+0000 7f73eeffd640 1 -- 10.20.193.121:0/2407092200 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f73f400b840 con 0x7f7400077f40 2026-02-13T10:39:23.252 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.243+0000 7f73eeffd640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f73dc077340 0x7f73dc079800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.253 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.243+0000 7f7405b8e640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f73dc077340 0x7f73dc079800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.253 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.243+0000 7f73eeffd640 1 -- 10.20.193.121:0/2407092200 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f73f4099430 con 0x7f7400077f40 2026-02-13T10:39:23.253 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.243+0000 7f7405b8e640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f73dc077340 0x7f73dc079800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73f00097c0 tx=0x7f73f00385b0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.253 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.245+0000 7f73eeffd640 1 -- 10.20.193.121:0/2407092200 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f73f401e070 con 0x7f7400077f40 2026-02-13T10:39:23.257 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:23.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:22 trial163 ceph-mon[24184]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:23.299 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- 10.20.193.121:0/4162061915 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4108660 msgr2=0x7f57f4108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.299 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 --2- 10.20.193.121:0/4162061915 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4108660 0x7f57f4108a40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f57e40099b0 tx=0x7f57e402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:23.299 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- 10.20.193.121:0/4162061915 shutdown_connections 2026-02-13T10:39:23.300 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 --2- 10.20.193.121:0/4162061915 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f57f4102660 0x7f57f4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.300 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 --2- 10.20.193.121:0/4162061915 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4108660 0x7f57f4108a40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.300 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- 10.20.193.121:0/4162061915 >> 10.20.193.121:0/4162061915 conn(0x7f57f40fe380 msgr2=0x7f57f41007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.300 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- 10.20.193.121:0/4162061915 shutdown_connections 2026-02-13T10:39:23.300 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- 10.20.193.121:0/4162061915 wait complete. 2026-02-13T10:39:23.302 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 Processor -- start 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- start start 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4102660 0x7f57f4075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f57f4108660 0x7f57f40758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f57f410f9c0 con 0x7f57f4102660 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57faac2640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f57f410f840 con 0x7f57f4108660 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57f8837640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4102660 0x7f57f4075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57f8837640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4102660 0x7f57f4075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50594/0 (socket says 10.20.193.121:50594) 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57f8837640 1 -- 10.20.193.121:0/3796839908 learned_addr learned my addr 10.20.193.121:0/3796839908 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57ebfff640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f57f4108660 0x7f57f40758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.303 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57f8837640 1 -- 10.20.193.121:0/3796839908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f57f4108660 msgr2=0x7f57f40758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57f8837640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f57f4108660 0x7f57f40758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.298+0000 7f57f8837640 1 -- 10.20.193.121:0/3796839908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4075df0 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.299+0000 7f57f8837640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4102660 0x7f57f4075370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f57e4002410 tx=0x7f57e4002960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.299+0000 7f57e9ffb640 1 -- 10.20.193.121:0/3796839908 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f57e4004210 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.299+0000 7f57e9ffb640 1 -- 10.20.193.121:0/3796839908 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f57e40043b0 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.299+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f4076080 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.299+0000 7f57e9ffb640 1 -- 10.20.193.121:0/3796839908 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f57e40419b0 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.299+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f41a6b80 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.299+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f4103da0 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.300+0000 7f57e9ffb640 1 -- 10.20.193.121:0/3796839908 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f57e4038770 con 0x7f57f4102660 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.300+0000 7f57e9ffb640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f57d00775a0 0x7f57d0079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.304 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.300+0000 7f57e9ffb640 1 -- 10.20.193.121:0/3796839908 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f57e403e070 con 0x7f57f4102660 2026-02-13T10:39:23.305 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.300+0000 7f57ebfff640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f57d00775a0 0x7f57d0079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.305 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.301+0000 7f57ebfff640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f57d00775a0 0x7f57d0079a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f57dc005f10 tx=0x7f57dc005ea0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.305 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.302+0000 7f57e9ffb640 1 -- 10.20.193.121:0/3796839908 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f57e4004560 con 0x7f57f4102660 2026-02-13T10:39:23.330 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.328+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efc2019faa0 con 0x7efc20102660 2026-02-13T10:39:23.330 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.328+0000 7efbfffff640 1 -- 10.20.193.121:0/3726685448 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc1406b9c0 con 0x7efc20102660 2026-02-13T10:39:23.330 INFO:teuthology.orchestra.run.trial121.stdout:137438953478 2026-02-13T10:39:23.332 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7efbf0077540 msgr2=0x7efbf0079a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.332 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7efbf0077540 0x7efbf0079a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc20100b40 tx=0x7efc0803a040 comp rx=0 tx=0).stop 2026-02-13T10:39:23.332 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 msgr2=0x7efc2019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.332 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc2019a6a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc1400d8d0 tx=0x7efc1400dda0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.332 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 shutdown_connections 2026-02-13T10:39:23.332 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7efbf0077540 0x7efbf0079a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.333 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7efc20108660 0x7efc2019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.333 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 --2- 10.20.193.121:0/3726685448 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7efc20102660 0x7efc2019a6a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.333 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 >> 10.20.193.121:0/3726685448 conn(0x7efc200fe380 msgr2=0x7efc200fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.333 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 shutdown_connections 2026-02-13T10:39:23.333 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.330+0000 7efc2541c640 1 -- 10.20.193.121:0/3726685448 wait complete. 2026-02-13T10:39:23.335 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- 10.20.193.121:0/3103908927 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c1048c0 msgr2=0x7f713c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.335 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 --2- 10.20.193.121:0/3103908927 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c1048c0 0x7f713c104d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f71300099b0 tx=0x7f713002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- 10.20.193.121:0/3103908927 shutdown_connections 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 --2- 10.20.193.121:0/3103908927 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c1048c0 0x7f713c104d20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 --2- 10.20.193.121:0/3103908927 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f713c10a8c0 0x7f713c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- 10.20.193.121:0/3103908927 >> 10.20.193.121:0/3103908927 conn(0x7f713c1005c0 msgr2=0x7f713c1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- 10.20.193.121:0/3103908927 shutdown_connections 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- 10.20.193.121:0/3103908927 wait complete. 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 Processor -- start 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- start start 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f713c1048c0 0x7f713c1a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c10a8c0 0x7f713c1a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.336 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f713c111ca0 con 0x7f713c10a8c0 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f71438bb640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f713c111b20 con 0x7f713c1048c0 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f7140e2f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c10a8c0 0x7f713c1a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f7140e2f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c10a8c0 0x7f713c1a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50602/0 (socket says 10.20.193.121:50602) 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f7140e2f640 1 -- 10.20.193.121:0/630977187 learned_addr learned my addr 10.20.193.121:0/630977187 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.332+0000 7f7141630640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f713c1048c0 0x7f713c1a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f7140e2f640 1 -- 10.20.193.121:0/630977187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f713c1048c0 msgr2=0x7f713c1a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f7140e2f640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f713c1048c0 0x7f713c1a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f7140e2f640 1 -- 10.20.193.121:0/630977187 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c19c880 con 0x7f713c10a8c0 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f7140e2f640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c10a8c0 0x7f713c1a2c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7130009980 tx=0x7f7130002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f712a7fc640 1 -- 10.20.193.121:0/630977187 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7130004420 con 0x7f713c10a8c0 2026-02-13T10:39:23.337 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c19cb10 con 0x7f713c10a8c0 2026-02-13T10:39:23.338 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f712a7fc640 1 -- 10.20.193.121:0/630977187 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f7130038c00 con 0x7f713c10a8c0 2026-02-13T10:39:23.338 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f712a7fc640 1 -- 10.20.193.121:0/630977187 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7130041ab0 con 0x7f713c10a8c0 2026-02-13T10:39:23.338 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c19d040 con 0x7f713c10a8c0 2026-02-13T10:39:23.338 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713c106000 con 0x7f713c10a8c0 2026-02-13T10:39:23.338 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.333+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff53819d220 con 0x7ff53810a8c0 2026-02-13T10:39:23.338 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.334+0000 7ff52dffb640 1 -- 10.20.193.121:0/4000109276 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff52008b6a0 con 0x7ff53810a8c0 2026-02-13T10:39:23.339 INFO:teuthology.orchestra.run.trial121.stdout:60129542157 2026-02-13T10:39:23.339 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.334+0000 7f712a7fc640 1 -- 10.20.193.121:0/630977187 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f7130041c90 con 0x7f713c10a8c0 2026-02-13T10:39:23.339 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.335+0000 7f712a7fc640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f71180775a0 0x7f7118079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.340 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.335+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f74001a2140 con 0x7f7400077f40 2026-02-13T10:39:23.340 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.335+0000 7f712a7fc640 1 -- 10.20.193.121:0/630977187 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f713003e070 con 0x7f713c10a8c0 2026-02-13T10:39:23.340 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.335+0000 7f7141630640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f71180775a0 0x7f7118079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.341 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.335+0000 7f7141630640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f71180775a0 0x7f7118079a60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f713c19d9c0 tx=0x7f712c008040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.341 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff5140775a0 msgr2=0x7ff514079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff5140775a0 0x7ff514079a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff528002930 tx=0x7ff528005f50 comp rx=0 tx=0).stop 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff53810a8c0 msgr2=0x7ff5381a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff53810a8c0 0x7ff5381a2b70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff520002410 tx=0x7ff5200028f0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 shutdown_connections 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7ff5140775a0 0x7ff514079a60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7ff53810a8c0 0x7ff5381a2b70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 --2- 10.20.193.121:0/4000109276 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7ff5381048c0 0x7ff5381a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 >> 10.20.193.121:0/4000109276 conn(0x7ff5381005c0 msgr2=0x7ff5381016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 shutdown_connections 2026-02-13T10:39:23.342 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.336+0000 7ff53ee6f640 1 -- 10.20.193.121:0/4000109276 wait complete. 2026-02-13T10:39:23.343 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.337+0000 7f73eeffd640 1 -- 10.20.193.121:0/2407092200 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f73f4099090 con 0x7f7400077f40 2026-02-13T10:39:23.343 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.337+0000 7f712a7fc640 1 -- 10.20.193.121:0/630977187 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7130035320 con 0x7f713c10a8c0 2026-02-13T10:39:23.343 INFO:teuthology.orchestra.run.trial121.stdout:111669149703 2026-02-13T10:39:23.344 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f73dc077340 msgr2=0x7f73dc079800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.344 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f73dc077340 0x7f73dc079800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f73f00097c0 tx=0x7f73f00385b0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.344 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077f40 msgr2=0x7f74001a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.344 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077f40 0x7f74001a1320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f73f400d8d0 tx=0x7f73f400dda0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.344 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 shutdown_connections 2026-02-13T10:39:23.345 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f73dc077340 0x7f73dc079800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.345 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7400077f40 0x7f74001a1320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.345 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 --2- 10.20.193.121:0/2407092200 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7400077620 0x7f74001a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.345 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 >> 10.20.193.121:0/2407092200 conn(0x7f74001005c0 msgr2=0x7f7400101220 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.345 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 shutdown_connections 2026-02-13T10:39:23.345 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.339+0000 7f7407e19640 1 -- 10.20.193.121:0/2407092200 wait complete. 2026-02-13T10:39:23.361 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.359+0000 7f518026f640 1 -- 10.20.193.121:0/4123498029 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077f40 msgr2=0x7f5178113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.361 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.359+0000 7f518026f640 1 --2- 10.20.193.121:0/4123498029 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077f40 0x7f5178113220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f516c0098e0 tx=0x7f516c02f1d0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.361 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.359+0000 7f518026f640 1 -- 10.20.193.121:0/4123498029 shutdown_connections 2026-02-13T10:39:23.361 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.359+0000 7f518026f640 1 --2- 10.20.193.121:0/4123498029 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077f40 0x7f5178113220 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.361 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.359+0000 7f518026f640 1 --2- 10.20.193.121:0/4123498029 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077620 0x7f5178077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.361 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.359+0000 7f518026f640 1 -- 10.20.193.121:0/4123498029 >> 10.20.193.121:0/4123498029 conn(0x7f51781005c0 msgr2=0x7f51781029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.361 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.359+0000 7f518026f640 1 -- 10.20.193.121:0/4123498029 shutdown_connections 2026-02-13T10:39:23.362 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.361+0000 7f518026f640 1 -- 10.20.193.121:0/4123498029 wait complete. 2026-02-13T10:39:23.365 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.361+0000 7f518026f640 1 Processor -- start 2026-02-13T10:39:23.365 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.361+0000 7f518026f640 1 -- start start 2026-02-13T10:39:23.365 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f518026f640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077620 0x7f51781a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.365 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f518026f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077f40 0x7f51781a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.365 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f518026f640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5178116170 con 0x7f5178077f40 2026-02-13T10:39:23.365 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f518026f640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5178115ff0 con 0x7f5178077620 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517d7e3640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077f40 0x7f51781a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517d7e3640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077f40 0x7f51781a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50618/0 (socket says 10.20.193.121:50618) 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517d7e3640 1 -- 10.20.193.121:0/3555125216 learned_addr learned my addr 10.20.193.121:0/3555125216 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517dfe4640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077620 0x7f51781a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517d7e3640 1 -- 10.20.193.121:0/3555125216 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077620 msgr2=0x7f51781a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517d7e3640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077620 0x7f51781a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517d7e3640 1 -- 10.20.193.121:0/3555125216 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51781a56e0 con 0x7f5178077f40 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f517d7e3640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077f40 0x7f51781a1350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f516c004600 tx=0x7f516c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f5166ffd640 1 -- 10.20.193.121:0/3555125216 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f516c046070 con 0x7f5178077f40 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51781a5970 con 0x7f5178077f40 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f5166ffd640 1 -- 10.20.193.121:0/3555125216 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f516c038510 con 0x7f5178077f40 2026-02-13T10:39:23.366 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51781a5ea0 con 0x7f5178077f40 2026-02-13T10:39:23.367 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.362+0000 7f5166ffd640 1 -- 10.20.193.121:0/3555125216 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f516c041630 con 0x7f5178077f40 2026-02-13T10:39:23.367 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.363+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5178078d60 con 0x7f5178077f40 2026-02-13T10:39:23.367 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.363+0000 7f5166ffd640 1 -- 10.20.193.121:0/3555125216 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f516c0386c0 con 0x7f5178077f40 2026-02-13T10:39:23.367 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.364+0000 7f5166ffd640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f51540775a0 0x7f5154079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.368 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.364+0000 7f5166ffd640 1 -- 10.20.193.121:0/3555125216 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f516c03e070 con 0x7f5178077f40 2026-02-13T10:39:23.368 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.364+0000 7f517dfe4640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f51540775a0 0x7f5154079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.368 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.365+0000 7f517dfe4640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f51540775a0 0x7f5154079a60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5168005fd0 tx=0x7f5168004420 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.368 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.367+0000 7f5166ffd640 1 -- 10.20.193.121:0/3555125216 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f516c08b0b0 con 0x7f5178077f40 2026-02-13T10:39:23.375 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953478 got 137438953478 for osd.4 2026-02-13T10:39:23.375 DEBUG:teuthology.parallel:result is None 2026-02-13T10:39:23.385 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542157 got 60129542157 for osd.1 2026-02-13T10:39:23.386 DEBUG:teuthology.parallel:result is None 2026-02-13T10:39:23.388 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149703 got 111669149703 for osd.3 2026-02-13T10:39:23.389 DEBUG:teuthology.parallel:result is None 2026-02-13T10:39:23.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.394+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f57f4105310 con 0x7f57f4102660 2026-02-13T10:39:23.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.394+0000 7f57e9ffb640 1 -- 10.20.193.121:0/3796839908 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f57e408b970 con 0x7f57f4102660 2026-02-13T10:39:23.396 INFO:teuthology.orchestra.run.trial121.stdout:77309411339 2026-02-13T10:39:23.398 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f57d00775a0 msgr2=0x7f57d0079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.398 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f57d00775a0 0x7f57d0079a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f57dc005f10 tx=0x7f57dc005ea0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.398 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4102660 msgr2=0x7f57f4075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4102660 0x7f57f4075370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f57e4002410 tx=0x7f57e4002960 comp rx=0 tx=0).stop 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 shutdown_connections 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f57d00775a0 0x7f57d0079a60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f57f4108660 0x7f57f40758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 --2- 10.20.193.121:0/3796839908 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f57f4102660 0x7f57f4075370 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 >> 10.20.193.121:0/3796839908 conn(0x7f57f40fe380 msgr2=0x7f57f4100500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 shutdown_connections 2026-02-13T10:39:23.399 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.396+0000 7f57faac2640 1 -- 10.20.193.121:0/3796839908 wait complete. 2026-02-13T10:39:23.430 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.428+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f713c19de40 con 0x7f713c10a8c0 2026-02-13T10:39:23.430 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.428+0000 7f712a7fc640 1 -- 10.20.193.121:0/630977187 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f713008b760 con 0x7f713c10a8c0 2026-02-13T10:39:23.430 INFO:teuthology.orchestra.run.trial121.stdout:154618822660 2026-02-13T10:39:23.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f71180775a0 msgr2=0x7f7118079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.432 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f71180775a0 0x7f7118079a60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f713c19d9c0 tx=0x7f712c008040 comp rx=0 tx=0).stop 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c10a8c0 msgr2=0x7f713c1a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c10a8c0 0x7f713c1a2c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7130009980 tx=0x7f7130002e20 comp rx=0 tx=0).stop 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 shutdown_connections 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f71180775a0 0x7f7118079a60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f713c10a8c0 0x7f713c1a2c00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 --2- 10.20.193.121:0/630977187 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f713c1048c0 0x7f713c1a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 >> 10.20.193.121:0/630977187 conn(0x7f713c1005c0 msgr2=0x7f713c1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 shutdown_connections 2026-02-13T10:39:23.433 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.430+0000 7f71438bb640 1 -- 10.20.193.121:0/630977187 wait complete. 2026-02-13T10:39:23.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411339 got 77309411339 for osd.2 2026-02-13T10:39:23.437 DEBUG:teuthology.parallel:result is None 2026-02-13T10:39:23.463 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.461+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f51781a21a0 con 0x7f5178077f40 2026-02-13T10:39:23.463 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.461+0000 7f5166ffd640 1 -- 10.20.193.121:0/3555125216 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f51781a21a0 con 0x7f5178077f40 2026-02-13T10:39:23.463 INFO:teuthology.orchestra.run.trial121.stdout:42949672975 2026-02-13T10:39:23.465 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f51540775a0 msgr2=0x7f5154079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.465 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f51540775a0 0x7f5154079a60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5168005fd0 tx=0x7f5168004420 comp rx=0 tx=0).stop 2026-02-13T10:39:23.465 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077f40 msgr2=0x7f51781a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.465 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077f40 0x7f51781a1350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f516c004600 tx=0x7f516c004630 comp rx=0 tx=0).stop 2026-02-13T10:39:23.465 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 shutdown_connections 2026-02-13T10:39:23.465 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f51540775a0 0x7f5154079a60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.466 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5178077f40 0x7f51781a1350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.466 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 --2- 10.20.193.121:0/3555125216 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5178077620 0x7f51781a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.466 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 >> 10.20.193.121:0/3555125216 conn(0x7f51781005c0 msgr2=0x7f51781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.466 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 shutdown_connections 2026-02-13T10:39:23.466 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.463+0000 7f518026f640 1 -- 10.20.193.121:0/3555125216 wait complete. 2026-02-13T10:39:23.469 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822660 got 154618822660 for osd.5 2026-02-13T10:39:23.469 DEBUG:teuthology.parallel:result is None 2026-02-13T10:39:23.501 INFO:tasks.cephadm.ceph_manager.ceph:need seq 42949672975 got 42949672975 for osd.0 2026-02-13T10:39:23.501 DEBUG:teuthology.parallel:result is None 2026-02-13T10:39:23.501 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2026-02-13T10:39:23.501 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph pg dump --format=json 2026-02-13T10:39:23.824 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:23.927 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- 10.20.193.121:0/1757765478 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 msgr2=0x7f550c104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.927 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 --2- 10.20.193.121:0/1757765478 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c104d10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f54f40099b0 tx=0x7f54f402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:23.927 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- 10.20.193.121:0/1757765478 shutdown_connections 2026-02-13T10:39:23.927 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 --2- 10.20.193.121:0/1757765478 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c104d10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.927 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 --2- 10.20.193.121:0/1757765478 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f550c10a8b0 0x7f550c10ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- 10.20.193.121:0/1757765478 >> 10.20.193.121:0/1757765478 conn(0x7f550c100590 msgr2=0x7f550c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- 10.20.193.121:0/1757765478 shutdown_connections 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- 10.20.193.121:0/1757765478 wait complete. 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 Processor -- start 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- start start 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c1a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f550c10a8b0 0x7f550c1a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f550c111c10 con 0x7f550c1048b0 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f55118a7640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f550c111a90 con 0x7f550c10a8b0 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f550affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c1a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.928 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f550affd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c1a2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50622/0 (socket says 10.20.193.121:50622) 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.924+0000 7f550affd640 1 -- 10.20.193.121:0/1747761043 learned_addr learned my addr 10.20.193.121:0/1747761043 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f550a7fc640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f550c10a8b0 0x7f550c1a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f550affd640 1 -- 10.20.193.121:0/1747761043 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f550c10a8b0 msgr2=0x7f550c1a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f550affd640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f550c10a8b0 0x7f550c1a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f550affd640 1 -- 10.20.193.121:0/1747761043 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c19c720 con 0x7f550c1048b0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f550affd640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c1a2650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f550000d8d0 tx=0x7f550000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f55108a5640 1 -- 10.20.193.121:0/1747761043 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5500014070 con 0x7f550c1048b0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f55108a5640 1 -- 10.20.193.121:0/1747761043 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f55000044a0 con 0x7f550c1048b0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c19ca10 con 0x7f550c1048b0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f55108a5640 1 -- 10.20.193.121:0/1747761043 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5500002e60 con 0x7f550c1048b0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c19cf70 con 0x7f550c1048b0 2026-02-13T10:39:23.929 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.925+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f550c105ff0 con 0x7f550c1048b0 2026-02-13T10:39:23.930 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.926+0000 7f55108a5640 1 -- 10.20.193.121:0/1747761043 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f550000b840 con 0x7f550c1048b0 2026-02-13T10:39:23.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.926+0000 7f55108a5640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f54e0077540 0x7f54e0079a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:23.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.926+0000 7f550a7fc640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f54e0077540 0x7f54e0079a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:23.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.926+0000 7f55108a5640 1 -- 10.20.193.121:0/1747761043 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f5500099720 con 0x7f550c1048b0 2026-02-13T10:39:23.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.927+0000 7f550a7fc640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f54e0077540 0x7f54e0079a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f550c19df70 tx=0x7f54f403a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:23.931 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:23.928+0000 7f55108a5640 1 -- 10.20.193.121:0/1747761043 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5500066bb0 con 0x7f550c1048b0 2026-02-13T10:39:24.011 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.009+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f550c19d290 con 0x7f54e0077540 2026-02-13T10:39:24.012 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.010+0000 7f55108a5640 1 -- 10.20.193.121:0/1747761043 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12335 (secure 0 0 0) 0x7f54f404b040 con 0x7f54e0077540 2026-02-13T10:39:24.012 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:24.013 INFO:teuthology.orchestra.run.trial121.stderr:dumped all 2026-02-13T10:39:24.013 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f54e0077540 msgr2=0x7f54e0079a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.013 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f54e0077540 0x7f54e0079a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f550c19df70 tx=0x7f54f403a040 comp rx=0 tx=0).stop 2026-02-13T10:39:24.013 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 msgr2=0x7f550c1a2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c1a2650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f550000d8d0 tx=0x7f550000dda0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 shutdown_connections 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f54e0077540 0x7f54e0079a00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f550c10a8b0 0x7f550c1a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 --2- 10.20.193.121:0/1747761043 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f550c1048b0 0x7f550c1a2650 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 >> 10.20.193.121:0/1747761043 conn(0x7f550c100590 msgr2=0x7f550c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 shutdown_connections 2026-02-13T10:39:24.014 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.011+0000 7f55118a7640 1 -- 10.20.193.121:0/1747761043 wait complete. 2026-02-13T10:39:24.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:23 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3726685448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2026-02-13T10:39:24.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:23 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/4000109276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2026-02-13T10:39:24.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:23 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2407092200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2026-02-13T10:39:24.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:23 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3796839908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2026-02-13T10:39:24.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:23 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/630977187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2026-02-13T10:39:24.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:23 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/3555125216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2026-02-13T10:39:24.048 INFO:teuthology.orchestra.run.trial121.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2026-02-13T10:39:23.558308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":4404019200,"kb_used":165528,"kb_used_data":3708,"kb_used_omap":9,"kb_used_meta":161654,"kb_avail":4403853672,"statfs":{"total":4509715660800,"available":4509546160128,"internally_reserved":0,"allocated":3796992,"data_stored":2553150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":165534374},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000973"},"pg_stats":[{"pgid":"1.0","version":"20'92","reported_seq":157,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-13T10:39:17.252910+0000","last_change":"2026-02-13T10:39:04.809120+0000","last_active":"2026-02-13T10:39:17.252910+0000","last_peered":"2026-02-13T10:39:17.252910+0000","last_clean":"2026-02-13T10:39:17.252910+0000","last_became_active":"2026-02-13T10:39:04.808951+0000","last_became_peered":"2026-02-13T10:39:04.808951+0000","last_unstale":"2026-02-13T10:39:17.252910+0000","last_undegraded":"2026-02-13T10:39:17.252910+0000","last_fullsized":"2026-02-13T10:39:17.252910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T10:38:43.614938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T10:38:43.614938+0000","last_clean_scrub_stamp":"2026-02-13T10:38:43.614938+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T15:50:31.682152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":36,"seq":154618822661,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975904,"statfs":{"total":751619276800,"available":751591325696,"internally_reserved":0,"allocated":335872,"data_stored":130341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27304,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975896,"statfs":{"total":751619276800,"available":751591317504,"internally_reserved":0,"allocated":335872,"data_stored":130341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27880,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975320,"statfs":{"total":751619276800,"available":751590727680,"internally_reserved":0,"allocated":929792,"data_stored":720709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411339,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975904,"statfs":{"total":751619276800,"available":751591325696,"internally_reserved":0,"allocated":335872,"data_stored":130341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27876,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975324,"statfs":{"total":751619276800,"available":751590731776,"internally_reserved":0,"allocated":929792,"data_stored":720709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27876,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975324,"statfs":{"total":751619276800,"available":751590731776,"internally_reserved":0,"allocated":929792,"data_stored":720709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-13T10:39:24.049 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph pg dump --format=json 2026-02-13T10:39:24.127 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:23 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3726685448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2026-02-13T10:39:24.127 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:23 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/4000109276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2026-02-13T10:39:24.127 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:23 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2407092200' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2026-02-13T10:39:24.127 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:23 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3796839908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2026-02-13T10:39:24.127 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:23 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/630977187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2026-02-13T10:39:24.127 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:23 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/3555125216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2026-02-13T10:39:24.337 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:24.444 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 -- 10.20.193.121:0/1735661867 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 msgr2=0x7f63dc077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 --2- 10.20.193.121:0/1735661867 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc077a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0099b0 tx=0x7f63cc02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 -- 10.20.193.121:0/1735661867 shutdown_connections 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 --2- 10.20.193.121:0/1735661867 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f63dc077f40 0x7f63dc113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 --2- 10.20.193.121:0/1735661867 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc077a00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 -- 10.20.193.121:0/1735661867 >> 10.20.193.121:0/1735661867 conn(0x7f63dc1005c0 msgr2=0x7f63dc1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 -- 10.20.193.121:0/1735661867 shutdown_connections 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 -- 10.20.193.121:0/1735661867 wait complete. 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 Processor -- start 2026-02-13T10:39:24.445 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.441+0000 7f63e2f23640 1 -- start start 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e2f23640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e2f23640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f63dc077f40 0x7f63dc1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e2f23640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc116160 con 0x7f63dc077620 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e2f23640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc115fe0 con 0x7f63dc077f40 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e0c98640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e0c98640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc1a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50646/0 (socket says 10.20.193.121:50646) 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e0c98640 1 -- 10.20.193.121:0/2081667459 learned_addr learned my addr 10.20.193.121:0/2081667459 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e0c98640 1 -- 10.20.193.121:0/2081667459 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f63dc077f40 msgr2=0x7f63dc1a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e0c98640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f63dc077f40 0x7f63dc1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e0c98640 1 -- 10.20.193.121:0/2081667459 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc1a57d0 con 0x7f63dc077620 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e0c98640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc1a0f00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63cc002410 tx=0x7f63cc0028f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:24.446 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63d1ffb640 1 -- 10.20.193.121:0/2081667459 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f63cc004210 con 0x7f63dc077620 2026-02-13T10:39:24.447 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc1a5a60 con 0x7f63dc077620 2026-02-13T10:39:24.447 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63d1ffb640 1 -- 10.20.193.121:0/2081667459 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f63cc0043b0 con 0x7f63dc077620 2026-02-13T10:39:24.447 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63d1ffb640 1 -- 10.20.193.121:0/2081667459 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f63cc041870 con 0x7f63dc077620 2026-02-13T10:39:24.447 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc1a5f90 con 0x7f63dc077620 2026-02-13T10:39:24.447 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.442+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc078d60 con 0x7f63dc077620 2026-02-13T10:39:24.448 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.443+0000 7f63d1ffb640 1 -- 10.20.193.121:0/2081667459 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f63cc041b70 con 0x7f63dc077620 2026-02-13T10:39:24.448 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.444+0000 7f63d1ffb640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f63b8077550 0x7f63b8079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:24.448 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.444+0000 7f63d1ffb640 1 -- 10.20.193.121:0/2081667459 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f63cc03e070 con 0x7f63dc077620 2026-02-13T10:39:24.448 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.444+0000 7f63d3fff640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f63b8077550 0x7f63b8079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:24.448 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.444+0000 7f63d3fff640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f63b8077550 0x7f63b8079a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005fd0 tx=0x7f63c4005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:24.448 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.445+0000 7f63d1ffb640 1 -- 10.20.193.121:0/2081667459 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f63cc049340 con 0x7f63dc077620 2026-02-13T10:39:24.527 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.525+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63dc1a6240 con 0x7f63b8077550 2026-02-13T10:39:24.527 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63d1ffb640 1 -- 10.20.193.121:0/2081667459 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12335 (secure 0 0 0) 0x7f63dc1a6240 con 0x7f63b8077550 2026-02-13T10:39:24.528 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:dumped all 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f63b8077550 msgr2=0x7f63b8079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f63b8077550 0x7f63b8079a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005fd0 tx=0x7f63c4005950 comp rx=0 tx=0).stop 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 msgr2=0x7f63dc1a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc1a0f00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63cc002410 tx=0x7f63cc0028f0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 shutdown_connections 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f63b8077550 0x7f63b8079a10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f63dc077f40 0x7f63dc1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.529 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.526+0000 7f63e2f23640 1 --2- 10.20.193.121:0/2081667459 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f63dc077620 0x7f63dc1a0f00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.530 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.527+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 >> 10.20.193.121:0/2081667459 conn(0x7f63dc1005c0 msgr2=0x7f63dc101360 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:24.530 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.527+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 shutdown_connections 2026-02-13T10:39:24.530 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.527+0000 7f63e2f23640 1 -- 10.20.193.121:0/2081667459 wait complete. 2026-02-13T10:39:24.565 INFO:teuthology.orchestra.run.trial121.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2026-02-13T10:39:23.558308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":4404019200,"kb_used":165528,"kb_used_data":3708,"kb_used_omap":9,"kb_used_meta":161654,"kb_avail":4403853672,"statfs":{"total":4509715660800,"available":4509546160128,"internally_reserved":0,"allocated":3796992,"data_stored":2553150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":165534374},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000973"},"pg_stats":[{"pgid":"1.0","version":"20'92","reported_seq":157,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-13T10:39:17.252910+0000","last_change":"2026-02-13T10:39:04.809120+0000","last_active":"2026-02-13T10:39:17.252910+0000","last_peered":"2026-02-13T10:39:17.252910+0000","last_clean":"2026-02-13T10:39:17.252910+0000","last_became_active":"2026-02-13T10:39:04.808951+0000","last_became_peered":"2026-02-13T10:39:04.808951+0000","last_unstale":"2026-02-13T10:39:17.252910+0000","last_undegraded":"2026-02-13T10:39:17.252910+0000","last_fullsized":"2026-02-13T10:39:17.252910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T10:38:43.614938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T10:38:43.614938+0000","last_clean_scrub_stamp":"2026-02-13T10:38:43.614938+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T15:50:31.682152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":36,"seq":154618822661,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975904,"statfs":{"total":751619276800,"available":751591325696,"internally_reserved":0,"allocated":335872,"data_stored":130341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27304,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975896,"statfs":{"total":751619276800,"available":751591317504,"internally_reserved":0,"allocated":335872,"data_stored":130341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27880,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975320,"statfs":{"total":751619276800,"available":751590727680,"internally_reserved":0,"allocated":929792,"data_stored":720709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411339,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":734003200,"kb_used":27296,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975904,"statfs":{"total":751619276800,"available":751591325696,"internally_reserved":0,"allocated":335872,"data_stored":130341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27876,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975324,"statfs":{"total":751619276800,"available":751590731776,"internally_reserved":0,"allocated":929792,"data_stored":720709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27876,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":733975324,"statfs":{"total":751619276800,"available":751590731776,"internally_reserved":0,"allocated":929792,"data_stored":720709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-13T10:39:24.566 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2026-02-13T10:39:24.566 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-13T10:39:24.566 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2026-02-13T10:39:24.567 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph health --format=json 2026-02-13T10:39:24.852 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:24.955 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 -- 10.20.193.121:0/408237182 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f26381048c0 msgr2=0x7f2638104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.955 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 --2- 10.20.193.121:0/408237182 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f26381048c0 0x7f2638104d20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f26280099b0 tx=0x7f262802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:24.955 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 -- 10.20.193.121:0/408237182 shutdown_connections 2026-02-13T10:39:24.955 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 --2- 10.20.193.121:0/408237182 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f26381048c0 0x7f2638104d20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.955 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 --2- 10.20.193.121:0/408237182 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f263810a8c0 0x7f263810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.955 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 -- 10.20.193.121:0/408237182 >> 10.20.193.121:0/408237182 conn(0x7f26381005c0 msgr2=0x7f26381029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:24.955 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 -- 10.20.193.121:0/408237182 shutdown_connections 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 -- 10.20.193.121:0/408237182 wait complete. 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 Processor -- start 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.951+0000 7f263e31e640 1 -- start start 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f263e31e640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f26381048c0 0x7f26381a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f263e31e640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f263810a8c0 0x7f26381a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f263e31e640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2638111c20 con 0x7f263810a8c0 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f263e31e640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2638111aa0 con 0x7f26381048c0 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26377fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f263810a8c0 0x7f26381a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26377fe640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f263810a8c0 0x7f26381a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50676/0 (socket says 10.20.193.121:50676) 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26377fe640 1 -- 10.20.193.121:0/2416872320 learned_addr learned my addr 10.20.193.121:0/2416872320 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:24.956 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26377fe640 1 -- 10.20.193.121:0/2416872320 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f26381048c0 msgr2=0x7f26381a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26377fe640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f26381048c0 0x7f26381a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26377fe640 1 -- 10.20.193.121:0/2416872320 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263819c730 con 0x7f263810a8c0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26377fe640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f263810a8c0 0x7f26381a2ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2628002410 tx=0x7f2628002960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26357fa640 1 -- 10.20.193.121:0/2416872320 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f2628004210 con 0x7f263810a8c0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26357fa640 1 -- 10.20.193.121:0/2416872320 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f26280043b0 con 0x7f263810a8c0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263819c9c0 con 0x7f263810a8c0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f26357fa640 1 -- 10.20.193.121:0/2416872320 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f26280419b0 con 0x7f263810a8c0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.952+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263819cec0 con 0x7f263810a8c0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.953+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2638106000 con 0x7f263810a8c0 2026-02-13T10:39:24.957 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.953+0000 7f26357fa640 1 -- 10.20.193.121:0/2416872320 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f2628038770 con 0x7f263810a8c0 2026-02-13T10:39:24.958 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.954+0000 7f26357fa640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f260c0775a0 0x7f260c079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:24.958 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.954+0000 7f2637fff640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f260c0775a0 0x7f260c079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:24.958 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.954+0000 7f26357fa640 1 -- 10.20.193.121:0/2416872320 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f262803e070 con 0x7f263810a8c0 2026-02-13T10:39:24.958 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.954+0000 7f2637fff640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f260c0775a0 0x7f260c079a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f262400a9f0 tx=0x7f2624005cf0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:24.959 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:24.955+0000 7f26357fa640 1 -- 10.20.193.121:0/2416872320 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2628004560 con 0x7f263810a8c0 2026-02-13T10:39:24.967 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:24 trial121 ceph-mon[19159]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:25.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f263819d170 con 0x7f263810a8c0 2026-02-13T10:39:25.056 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f26357fa640 1 -- 10.20.193.121:0/2416872320 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f262808baa0 con 0x7f263810a8c0 2026-02-13T10:39:25.057 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:25.057 INFO:teuthology.orchestra.run.trial121.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-13T10:39:25.058 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f260c0775a0 msgr2=0x7f260c079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:25.058 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f260c0775a0 0x7f260c079a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f262400a9f0 tx=0x7f2624005cf0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.058 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f263810a8c0 msgr2=0x7f26381a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:25.058 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f263810a8c0 0x7f26381a2ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2628002410 tx=0x7f2628002960 comp rx=0 tx=0).stop 2026-02-13T10:39:25.058 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 shutdown_connections 2026-02-13T10:39:25.059 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f260c0775a0 0x7f260c079a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.059 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f263810a8c0 0x7f26381a2ba0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.059 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 --2- 10.20.193.121:0/2416872320 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f26381048c0 0x7f26381a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.059 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.055+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 >> 10.20.193.121:0/2416872320 conn(0x7f26381005c0 msgr2=0x7f2638101660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:25.059 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.056+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 shutdown_connections 2026-02-13T10:39:25.059 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.056+0000 7f263e31e640 1 -- 10.20.193.121:0/2416872320 wait complete. 2026-02-13T10:39:25.098 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2026-02-13T10:39:25.098 INFO:tasks.cephadm:Setup complete, yielding 2026-02-13T10:39:25.099 INFO:teuthology.run_tasks:Running task print... 2026-02-13T10:39:25.112 INFO:teuthology.task.print:**** done end installing squid cephadm ... 2026-02-13T10:39:25.113 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-13T10:39:25.127 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:39:25.127 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2026-02-13T10:39:25.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:24 trial163 ceph-mon[24184]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:25.410 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:25.516 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.512+0000 7f35d08db640 1 -- 10.20.193.121:0/2010192244 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8100f10 msgr2=0x7f35c8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:25.516 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.512+0000 7f35d08db640 1 --2- 10.20.193.121:0/2010192244 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8100f10 0x7f35c8103320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f35c00099b0 tx=0x7f35c002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:25.516 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.512+0000 7f35d08db640 1 -- 10.20.193.121:0/2010192244 shutdown_connections 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.512+0000 7f35d08db640 1 --2- 10.20.193.121:0/2010192244 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f35c8103860 0x7f35c8105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.512+0000 7f35d08db640 1 --2- 10.20.193.121:0/2010192244 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8100f10 0x7f35c8103320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.512+0000 7f35d08db640 1 -- 10.20.193.121:0/2010192244 >> 10.20.193.121:0/2010192244 conn(0x7f35c80fcd20 msgr2=0x7f35c80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.512+0000 7f35d08db640 1 -- 10.20.193.121:0/2010192244 shutdown_connections 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35d08db640 1 -- 10.20.193.121:0/2010192244 wait complete. 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35d08db640 1 Processor -- start 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35d08db640 1 -- start start 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35d08db640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f35c8100f10 0x7f35c81a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35d08db640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8103860 0x7f35c81a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35d08db640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35c8111b90 con 0x7f35c8103860 2026-02-13T10:39:25.517 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35d08db640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f35c8111a10 con 0x7f35c8100f10 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35cde4f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8103860 0x7f35c81a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35cde4f640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8103860 0x7f35c81a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50686/0 (socket says 10.20.193.121:50686) 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35cde4f640 1 -- 10.20.193.121:0/957716569 learned_addr learned my addr 10.20.193.121:0/957716569 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35ce650640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f35c8100f10 0x7f35c81a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35cde4f640 1 -- 10.20.193.121:0/957716569 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f35c8100f10 msgr2=0x7f35c81a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35cde4f640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f35c8100f10 0x7f35c81a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.513+0000 7f35cde4f640 1 -- 10.20.193.121:0/957716569 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c81a15a0 con 0x7f35c8103860 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.514+0000 7f35cde4f640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8103860 0x7f35c81a0f20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b700 tx=0x7f35b400bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.514+0000 7f35b37fe640 1 -- 10.20.193.121:0/957716569 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f35b4004440 con 0x7f35c8103860 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.514+0000 7f35b37fe640 1 -- 10.20.193.121:0/957716569 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f35b4010070 con 0x7f35c8103860 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.514+0000 7f35b37fe640 1 -- 10.20.193.121:0/957716569 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f35b400caa0 con 0x7f35c8103860 2026-02-13T10:39:25.518 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.514+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c81a5fe0 con 0x7f35c8103860 2026-02-13T10:39:25.519 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.514+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c81a6560 con 0x7f35c8103860 2026-02-13T10:39:25.519 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.514+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c8101390 con 0x7f35c8103860 2026-02-13T10:39:25.519 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.515+0000 7f35b37fe640 1 -- 10.20.193.121:0/957716569 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f35b401d030 con 0x7f35c8103860 2026-02-13T10:39:25.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.515+0000 7f35b37fe640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f35a4077550 0x7f35a4079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:25.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.515+0000 7f35b37fe640 1 -- 10.20.193.121:0/957716569 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f35b4098570 con 0x7f35c8103860 2026-02-13T10:39:25.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.515+0000 7f35ce650640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f35a4077550 0x7f35a4079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:25.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.516+0000 7f35ce650640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f35a4077550 0x7f35a4079a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35c0002be0 tx=0x7f35c0005cb0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:25.520 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.517+0000 7f35b37fe640 1 -- 10.20.193.121:0/957716569 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f35b4065b10 con 0x7f35c8103860 2026-02-13T10:39:25.601 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.599+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) -- 0x7f35c81a6fb0 con 0x7f35c8103860 2026-02-13T10:39:25.603 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.602+0000 7f35b37fe640 1 -- 10.20.193.121:0/957716569 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v40)=0 v40) ==== 143+0+0 (secure 0 0 0) 0x7f35b406a9c0 con 0x7f35c8103860 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f35a4077550 msgr2=0x7f35a4079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f35a4077550 0x7f35a4079a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35c0002be0 tx=0x7f35c0005cb0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8103860 msgr2=0x7f35c81a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8103860 0x7f35c81a0f20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b700 tx=0x7f35b400bbd0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 shutdown_connections 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f35a4077550 0x7f35a4079a10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f35c8103860 0x7f35c81a0f20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 --2- 10.20.193.121:0/957716569 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f35c8100f10 0x7f35c81a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 >> 10.20.193.121:0/957716569 conn(0x7f35c80fcd20 msgr2=0x7f35c80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:25.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 shutdown_connections 2026-02-13T10:39:25.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:25.603+0000 7f35d08db640 1 -- 10.20.193.121:0/957716569 wait complete. 2026-02-13T10:39:25.642 INFO:teuthology.run_tasks:Running task print... 2026-02-13T10:39:25.656 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2026-02-13T10:39:25.656 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-13T10:39:25.670 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:39:25.671 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph orch status' 2026-02-13T10:39:25.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T10:39:25.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T10:39:25.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2416872320' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2026-02-13T10:39:25.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/957716569' entity='client.admin' 2026-02-13T10:39:25.958 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:25.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:25.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:25.959 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:25 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:25.959 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:26.065 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- 10.20.193.121:0/1117540176 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4105b40 msgr2=0x7f43f4105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/1117540176 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4105b40 0x7f43f4105fc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0099b0 tx=0x7f43dc02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- 10.20.193.121:0/1117540176 shutdown_connections 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/1117540176 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4105b40 0x7f43f4105fc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/1117540176 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f43f4104940 0x7f43f4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- 10.20.193.121:0/1117540176 >> 10.20.193.121:0/1117540176 conn(0x7f43f41000f0 msgr2=0x7f43f4102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- 10.20.193.121:0/1117540176 shutdown_connections 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- 10.20.193.121:0/1117540176 wait complete. 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 Processor -- start 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- start start 2026-02-13T10:39:26.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4104940 0x7f43f419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f43f4105b40 0x7f43f419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410d610 con 0x7f43f4104940 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f8fbd640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410d490 con 0x7f43f4105b40 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f2575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4104940 0x7f43f419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f2575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4104940 0x7f43f419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50706/0 (socket says 10.20.193.121:50706) 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f2575640 1 -- 10.20.193.121:0/421595972 learned_addr learned my addr 10.20.193.121:0/421595972 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f1d74640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f43f4105b40 0x7f43f419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f2575640 1 -- 10.20.193.121:0/421595972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f43f4105b40 msgr2=0x7f43f419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f2575640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f43f4105b40 0x7f43f419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f2575640 1 -- 10.20.193.121:0/421595972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f419cf90 con 0x7f43f4104940 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.062+0000 7f43f1d74640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f43f4105b40 0x7f43f419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:26.067 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.063+0000 7f43f2575640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4104940 0x7f43f419c3d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e960 tx=0x7f43e800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:26.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.063+0000 7f43db7fe640 1 -- 10.20.193.121:0/421595972 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f43e800cb80 con 0x7f43f4104940 2026-02-13T10:39:26.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.063+0000 7f43db7fe640 1 -- 10.20.193.121:0/421595972 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f43e8004560 con 0x7f43f4104940 2026-02-13T10:39:26.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.063+0000 7f43db7fe640 1 -- 10.20.193.121:0/421595972 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f43e8010430 con 0x7f43f4104940 2026-02-13T10:39:26.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.063+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f41a19d0 con 0x7f43f4104940 2026-02-13T10:39:26.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.063+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f41a1fe0 con 0x7f43f4104940 2026-02-13T10:39:26.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.063+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f4104d40 con 0x7f43f4104940 2026-02-13T10:39:26.068 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.064+0000 7f43db7fe640 1 -- 10.20.193.121:0/421595972 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f43e8020030 con 0x7f43f4104940 2026-02-13T10:39:26.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.065+0000 7f43db7fe640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f43cc0775a0 0x7f43cc079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:26.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.065+0000 7f43f1d74640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f43cc0775a0 0x7f43cc079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:26.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.065+0000 7f43db7fe640 1 -- 10.20.193.121:0/421595972 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f43e8099690 con 0x7f43f4104940 2026-02-13T10:39:26.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.065+0000 7f43f1d74640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f43cc0775a0 0x7f43cc079a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0097c0 tx=0x7f43dc005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:26.069 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.066+0000 7f43db7fe640 1 -- 10.20.193.121:0/421595972 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f43e8066bc0 con 0x7f43f4104940 2026-02-13T10:39:26.157 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.156+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f43f40630c0 con 0x7f43cc0775a0 2026-02-13T10:39:26.158 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43db7fe640 1 -- 10.20.193.121:0/421595972 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f43f40630c0 con 0x7f43cc0775a0 2026-02-13T10:39:26.158 INFO:teuthology.orchestra.run.trial121.stdout:Backend: cephadm 2026-02-13T10:39:26.159 INFO:teuthology.orchestra.run.trial121.stdout:Available: Yes 2026-02-13T10:39:26.159 INFO:teuthology.orchestra.run.trial121.stdout:Paused: No 2026-02-13T10:39:26.159 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f43cc0775a0 msgr2=0x7f43cc079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f43cc0775a0 0x7f43cc079a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0097c0 tx=0x7f43dc005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4104940 msgr2=0x7f43f419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4104940 0x7f43f419c3d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e960 tx=0x7f43e800ee30 comp rx=0 tx=0).stop 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 shutdown_connections 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f43cc0775a0 0x7f43cc079a60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f43f4105b40 0x7f43f419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 --2- 10.20.193.121:0/421595972 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f43f4104940 0x7f43f419c3d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 >> 10.20.193.121:0/421595972 conn(0x7f43f41000f0 msgr2=0x7f43f4109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 shutdown_connections 2026-02-13T10:39:26.160 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.157+0000 7f43f8fbd640 1 -- 10.20.193.121:0/421595972 wait complete. 2026-02-13T10:39:26.210 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph orch ps' 2026-02-13T10:39:26.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T10:39:26.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T10:39:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2416872320' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2026-02-13T10:39:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/957716569' entity='client.admin' 2026-02-13T10:39:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:26.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:25 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:26.499 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:26.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 -- 10.20.193.121:0/409377672 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4105b40 msgr2=0x7f04e4105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.605 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 --2- 10.20.193.121:0/409377672 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4105b40 0x7f04e4105fc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04d80099b0 tx=0x7f04d802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 -- 10.20.193.121:0/409377672 shutdown_connections 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 --2- 10.20.193.121:0/409377672 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4105b40 0x7f04e4105fc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 --2- 10.20.193.121:0/409377672 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f04e4104940 0x7f04e4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 -- 10.20.193.121:0/409377672 >> 10.20.193.121:0/409377672 conn(0x7f04e41000f0 msgr2=0x7f04e4102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 -- 10.20.193.121:0/409377672 shutdown_connections 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.601+0000 7f04ec264640 1 -- 10.20.193.121:0/409377672 wait complete. 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 Processor -- start 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 -- start start 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4104940 0x7f04e419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f04e4105b40 0x7f04e419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410d610 con 0x7f04e4104940 2026-02-13T10:39:26.606 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410d490 con 0x7f04e4105b40 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e9fd9640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4104940 0x7f04e419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e9fd9640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4104940 0x7f04e419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50724/0 (socket says 10.20.193.121:50724) 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e9fd9640 1 -- 10.20.193.121:0/3139138724 learned_addr learned my addr 10.20.193.121:0/3139138724 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e97d8640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f04e4105b40 0x7f04e419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e9fd9640 1 -- 10.20.193.121:0/3139138724 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f04e4105b40 msgr2=0x7f04e419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e9fd9640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f04e4105b40 0x7f04e419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e9fd9640 1 -- 10.20.193.121:0/3139138724 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e419cfc0 con 0x7f04e4104940 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04e9fd9640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4104940 0x7f04e419c400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c910 tx=0x7f04d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04d2ffd640 1 -- 10.20.193.121:0/3139138724 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f04d4007a70 con 0x7f04e4104940 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04d2ffd640 1 -- 10.20.193.121:0/3139138724 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f04d4007c10 con 0x7f04e4104940 2026-02-13T10:39:26.607 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04d2ffd640 1 -- 10.20.193.121:0/3139138724 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f04d400f780 con 0x7f04e4104940 2026-02-13T10:39:26.608 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41a1a00 con 0x7f04e4104940 2026-02-13T10:39:26.608 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.602+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a2010 con 0x7f04e4104940 2026-02-13T10:39:26.608 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.603+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e4104d40 con 0x7f04e4104940 2026-02-13T10:39:26.608 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.604+0000 7f04d2ffd640 1 -- 10.20.193.121:0/3139138724 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f04d400fa30 con 0x7f04e4104940 2026-02-13T10:39:26.609 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.604+0000 7f04d2ffd640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f04c0077550 0x7f04c0079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:26.609 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.604+0000 7f04d2ffd640 1 -- 10.20.193.121:0/3139138724 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f04d4099b80 con 0x7f04e4104940 2026-02-13T10:39:26.609 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.604+0000 7f04e97d8640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f04c0077550 0x7f04c0079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:26.609 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.604+0000 7f04e97d8640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f04c0077550 0x7f04c0079a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f04d8002be0 tx=0x7f04d803a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:26.609 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.606+0000 7f04d2ffd640 1 -- 10.20.193.121:0/3139138724 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f04d4067010 con 0x7f04e4104940 2026-02-13T10:39:26.694 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.692+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f04e40630c0 con 0x7f04c0077550 2026-02-13T10:39:26.698 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.697+0000 7f04d2ffd640 1 -- 10.20.193.121:0/3139138724 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3080 (secure 0 0 0) 0x7f04e40630c0 con 0x7f04c0077550 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:alertmanager.trial121 trial121 *:9093,9094 running (86s) 51s ago 110s 29.4M - 0.25.0 c8568f914cd2 b9d986ac7979 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:ceph-exporter.trial121 trial121 *:9926 running (114s) 51s ago 114s 8514k - 19.2.3-672-g283431c0 4cc07a41bb19 f5181d4e01d3 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:ceph-exporter.trial163 trial163 *:9926 running (98s) 18s ago 98s 6567k - 19.2.3-672-g283431c0 4cc07a41bb19 e2e7bd6f3641 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:crash.trial121 trial121 running (113s) 51s ago 113s 7646k - 19.2.3-672-g283431c0 4cc07a41bb19 1d92af35b7ec 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:crash.trial163 trial163 running (97s) 18s ago 97s 7650k - 19.2.3-672-g283431c0 4cc07a41bb19 9555aac72e98 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:grafana.trial121 trial121 *:3000 running (86s) 51s ago 107s 79.3M - 10.4.0 c8b91775d855 2ba53ebcec40 2026-02-13T10:39:26.699 INFO:teuthology.orchestra.run.trial121.stdout:mgr.trial121.tkoihk trial121 *:9283,8765,8443 running (2m) 51s ago 2m 542M - 19.2.3-672-g283431c0 4cc07a41bb19 20b8c5c28153 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:mgr.trial163.xfyaeo trial163 *:8443,9283,8765 running (95s) 18s ago 95s 489M - 19.2.3-672-g283431c0 4cc07a41bb19 ef90c6c01a71 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:mon.trial121 trial121 running (2m) 51s ago 2m 47.6M 2048M 19.2.3-672-g283431c0 4cc07a41bb19 c5ae73bd5b1b 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:mon.trial163 trial163 running (94s) 18s ago 94s 44.3M 2048M 19.2.3-672-g283431c0 4cc07a41bb19 645ee540e429 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:node-exporter.trial121 trial121 *:9100 running (112s) 51s ago 112s 11.9M - 1.7.0 72c9c2088986 c94c82f98d38 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:node-exporter.trial163 trial163 *:9100 running (96s) 18s ago 96s 11.4M - 1.7.0 72c9c2088986 2b3e29ca2b70 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:osd.0 trial121 running (74s) 51s ago 74s 61.4M 25.6G 19.2.3-672-g283431c0 4cc07a41bb19 379a9e166bb9 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:osd.1 trial121 running (63s) 51s ago 63s 60.9M 25.6G 19.2.3-672-g283431c0 4cc07a41bb19 2a29d84c07b3 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:osd.2 trial121 running (52s) 51s ago 52s 15.0M 25.6G 19.2.3-672-g283431c0 4cc07a41bb19 82f925ca8d68 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:osd.3 trial163 running (41s) 18s ago 41s 63.6M 26.6G 19.2.3-672-g283431c0 4cc07a41bb19 cbffba08a228 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:osd.4 trial163 running (30s) 18s ago 30s 61.9M 26.6G 19.2.3-672-g283431c0 4cc07a41bb19 749d9c037d1c 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:osd.5 trial163 running (19s) 18s ago 19s 13.2M 26.6G 19.2.3-672-g283431c0 4cc07a41bb19 09d4c15cba01 2026-02-13T10:39:26.700 INFO:teuthology.orchestra.run.trial121.stdout:prometheus.trial121 trial121 *:9095 running (85s) 51s ago 104s 38.6M - 2.51.0 1d3b7f56885b cf75721304a2 2026-02-13T10:39:26.701 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f04c0077550 msgr2=0x7f04c0079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.701 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f04c0077550 0x7f04c0079a10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f04d8002be0 tx=0x7f04d803a040 comp rx=0 tx=0).stop 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4104940 msgr2=0x7f04e419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4104940 0x7f04e419c400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c910 tx=0x7f04d400cde0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 shutdown_connections 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f04c0077550 0x7f04c0079a10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f04e4105b40 0x7f04e419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 --2- 10.20.193.121:0/3139138724 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f04e4104940 0x7f04e419c400 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 >> 10.20.193.121:0/3139138724 conn(0x7f04e41000f0 msgr2=0x7f04e4109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 shutdown_connections 2026-02-13T10:39:26.702 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:26.698+0000 7f04ec264640 1 -- 10.20.193.121:0/3139138724 wait complete. 2026-02-13T10:39:26.737 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph orch ls' 2026-02-13T10:39:27.023 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:27.134 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 -- 10.20.193.121:0/1619932448 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 msgr2=0x7f5a44077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.134 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 --2- 10.20.193.121:0/1619932448 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a44077ea0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5a340099b0 tx=0x7f5a3402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:27.134 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 -- 10.20.193.121:0/1619932448 shutdown_connections 2026-02-13T10:39:27.134 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 --2- 10.20.193.121:0/1619932448 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5a44078cf0 0x7f5a44079150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.134 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 --2- 10.20.193.121:0/1619932448 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a44077ea0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.134 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 -- 10.20.193.121:0/1619932448 >> 10.20.193.121:0/1619932448 conn(0x7f5a441000f0 msgr2=0x7f5a44102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:27.134 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 -- 10.20.193.121:0/1619932448 shutdown_connections 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.130+0000 7f5a4b610640 1 -- 10.20.193.121:0/1619932448 wait complete. 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 Processor -- start 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 -- start start 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a441a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5a44078cf0 0x7f5a441a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44111b20 con 0x7f5a44077aa0 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441119a0 con 0x7f5a44078cf0 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a49385640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a441a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a49385640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a441a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:50756/0 (socket says 10.20.193.121:50756) 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a49385640 1 -- 10.20.193.121:0/2181825469 learned_addr learned my addr 10.20.193.121:0/2181825469 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:27.135 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a49385640 1 -- 10.20.193.121:0/2181825469 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5a44078cf0 msgr2=0x7f5a441a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a49385640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5a44078cf0 0x7f5a441a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a49385640 1 -- 10.20.193.121:0/2181825469 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a441a14d0 con 0x7f5a44077aa0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a49385640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a441a0910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5a3402fd00 tx=0x7f5a3402fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a327fc640 1 -- 10.20.193.121:0/2181825469 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5a34004420 con 0x7f5a44077aa0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a327fc640 1 -- 10.20.193.121:0/2181825469 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f5a3402fe50 con 0x7f5a44077aa0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a441a5eb0 con 0x7f5a44077aa0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a327fc640 1 -- 10.20.193.121:0/2181825469 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f5a34041a00 con 0x7f5a44077aa0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.131+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a441a6430 con 0x7f5a44077aa0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.132+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a44077ea0 con 0x7f5a44077aa0 2026-02-13T10:39:27.136 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.132+0000 7f5a327fc640 1 -- 10.20.193.121:0/2181825469 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f5a34041ba0 con 0x7f5a44077aa0 2026-02-13T10:39:27.137 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.133+0000 7f5a327fc640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a200775a0 0x7f5a20079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:27.137 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.133+0000 7f5a48b84640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a200775a0 0x7f5a20079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:27.137 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.133+0000 7f5a327fc640 1 -- 10.20.193.121:0/2181825469 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f5a3403e070 con 0x7f5a44077aa0 2026-02-13T10:39:27.138 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.133+0000 7f5a48b84640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a200775a0 0x7f5a20079a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005fd0 tx=0x7f5a38005950 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:27.138 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.134+0000 7f5a327fc640 1 -- 10.20.193.121:0/2181825469 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5a34049340 con 0x7f5a44077aa0 2026-02-13T10:39:27.138 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:26 trial121 ceph-mon[19159]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:27.222 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.220+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5a440630c0 con 0x7f5a200775a0 2026-02-13T10:39:27.224 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.223+0000 7f5a327fc640 1 -- 10.20.193.121:0/2181825469 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1280 (secure 0 0 0) 0x7f5a440630c0 con 0x7f5a200775a0 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:alertmanager ?:9093,9094 1/1 52s ago 2m count:1 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:ceph-exporter ?:9926 2/2 52s ago 2m * 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:crash 2/2 52s ago 2m * 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:grafana ?:3000 1/1 52s ago 2m count:1 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:mgr 2/2 52s ago 2m count:2 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:mon 2/2 52s ago 110s trial121:10.20.193.121=trial121;trial163:10.20.193.163=trial163;count:2 2026-02-13T10:39:27.225 INFO:teuthology.orchestra.run.trial121.stdout:node-exporter ?:9100 2/2 52s ago 2m * 2026-02-13T10:39:27.226 INFO:teuthology.orchestra.run.trial121.stdout:osd 6 52s ago - 2026-02-13T10:39:27.226 INFO:teuthology.orchestra.run.trial121.stdout:prometheus ?:9095 1/1 52s ago 2m count:1 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a200775a0 msgr2=0x7f5a20079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a200775a0 0x7f5a20079a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005fd0 tx=0x7f5a38005950 comp rx=0 tx=0).stop 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 msgr2=0x7f5a441a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a441a0910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5a3402fd00 tx=0x7f5a3402fd30 comp rx=0 tx=0).stop 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 shutdown_connections 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f5a200775a0 0x7f5a20079a60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f5a44078cf0 0x7f5a441a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 --2- 10.20.193.121:0/2181825469 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f5a44077aa0 0x7f5a441a0910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.227 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 >> 10.20.193.121:0/2181825469 conn(0x7f5a441000f0 msgr2=0x7f5a4410f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:27.228 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 shutdown_connections 2026-02-13T10:39:27.228 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.224+0000 7f5a4b610640 1 -- 10.20.193.121:0/2181825469 wait complete. 2026-02-13T10:39:27.261 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph orch host ls' 2026-02-13T10:39:27.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:26 trial163 ceph-mon[24184]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:27.547 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:27.651 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 -- 10.20.193.121:0/2624064055 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1026b0 msgr2=0x7f825c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.651 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 --2- 10.20.193.121:0/2624064055 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1026b0 0x7f825c102ab0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f824c0099b0 tx=0x7f824c02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:27.651 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 -- 10.20.193.121:0/2624064055 shutdown_connections 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 --2- 10.20.193.121:0/2624064055 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1038b0 0x7f825c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 --2- 10.20.193.121:0/2624064055 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1026b0 0x7f825c102ab0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 -- 10.20.193.121:0/2624064055 >> 10.20.193.121:0/2624064055 conn(0x7f825c0fde60 msgr2=0x7f825c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 -- 10.20.193.121:0/2624064055 shutdown_connections 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.647+0000 7f82644ea640 1 -- 10.20.193.121:0/2624064055 wait complete. 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f82644ea640 1 Processor -- start 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f82644ea640 1 -- start start 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f82644ea640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1026b0 0x7f825c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f82644ea640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1038b0 0x7f825c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:27.652 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f82644ea640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10b3a0 con 0x7f825c1038b0 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f82644ea640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f825c10b220 con 0x7f825c1026b0 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f826225f640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1026b0 0x7f825c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f826225f640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1026b0 0x7f825c19a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.163:3300/0 says I am v2:10.20.193.121:42288/0 (socket says 10.20.193.121:42288) 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f826225f640 1 -- 10.20.193.121:0/2635971219 learned_addr learned my addr 10.20.193.121:0/2635971219 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.648+0000 7f8261a5e640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1038b0 0x7f825c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f8261a5e640 1 -- 10.20.193.121:0/2635971219 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1026b0 msgr2=0x7f825c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f8261a5e640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1026b0 0x7f825c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f8261a5e640 1 -- 10.20.193.121:0/2635971219 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c19ad20 con 0x7f825c1038b0 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f826225f640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1026b0 0x7f825c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f8261a5e640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1038b0 0x7f825c19a6a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f825000d6d0 tx=0x7f825000dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f82437fe640 1 -- 10.20.193.121:0/2635971219 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f8250014070 con 0x7f825c1038b0 2026-02-13T10:39:27.653 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f82437fe640 1 -- 10.20.193.121:0/2635971219 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f8250004d30 con 0x7f825c1038b0 2026-02-13T10:39:27.654 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c19f760 con 0x7f825c1038b0 2026-02-13T10:39:27.654 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f82437fe640 1 -- 10.20.193.121:0/2635971219 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f8250005020 con 0x7f825c1038b0 2026-02-13T10:39:27.654 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c19fd70 con 0x7f825c1038b0 2026-02-13T10:39:27.654 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.649+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c102ab0 con 0x7f825c1038b0 2026-02-13T10:39:27.654 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.650+0000 7f82437fe640 1 -- 10.20.193.121:0/2635971219 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f82500047c0 con 0x7f825c1038b0 2026-02-13T10:39:27.655 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.650+0000 7f82437fe640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f82380775a0 0x7f8238079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:27.655 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.650+0000 7f826225f640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f82380775a0 0x7f8238079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:27.655 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.650+0000 7f82437fe640 1 -- 10.20.193.121:0/2635971219 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f82500995b0 con 0x7f825c1038b0 2026-02-13T10:39:27.655 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.650+0000 7f826225f640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f82380775a0 0x7f8238079a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f824c0097c0 tx=0x7f824c005cb0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:27.655 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.652+0000 7f82437fe640 1 -- 10.20.193.121:0/2635971219 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8250066a20 con 0x7f825c1038b0 2026-02-13T10:39:27.738 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.736+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f825c0630c0 con 0x7f82380775a0 2026-02-13T10:39:27.739 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.737+0000 7f82437fe640 1 -- 10.20.193.121:0/2635971219 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+145 (secure 0 0 0) 0x7f825c0630c0 con 0x7f82380775a0 2026-02-13T10:39:27.739 INFO:teuthology.orchestra.run.trial121.stdout:HOST ADDR LABELS STATUS 2026-02-13T10:39:27.739 INFO:teuthology.orchestra.run.trial121.stdout:trial121 10.20.193.121 2026-02-13T10:39:27.739 INFO:teuthology.orchestra.run.trial121.stdout:trial163 10.20.193.163 2026-02-13T10:39:27.739 INFO:teuthology.orchestra.run.trial121.stdout:2 hosts in cluster 2026-02-13T10:39:27.740 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f82380775a0 msgr2=0x7f8238079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.740 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f82380775a0 0x7f8238079a60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f824c0097c0 tx=0x7f824c005cb0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.740 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1038b0 msgr2=0x7f825c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1038b0 0x7f825c19a6a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f825000d6d0 tx=0x7f825000dba0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 shutdown_connections 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f82380775a0 0x7f8238079a60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f825c1038b0 0x7f825c19a6a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 --2- 10.20.193.121:0/2635971219 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f825c1026b0 0x7f825c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 >> 10.20.193.121:0/2635971219 conn(0x7f825c0fde60 msgr2=0x7f825c106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 shutdown_connections 2026-02-13T10:39:27.741 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:27.738+0000 7f82644ea640 1 -- 10.20.193.121:0/2635971219 wait complete. 2026-02-13T10:39:27.777 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph orch device ls' 2026-02-13T10:39:28.063 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:28.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 -- 10.20.193.121:0/3302563939 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841038b0 msgr2=0x7fae84103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:28.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/3302563939 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841038b0 0x7fae84103d30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fae780099b0 tx=0x7fae7802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:28.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 -- 10.20.193.121:0/3302563939 shutdown_connections 2026-02-13T10:39:28.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/3302563939 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841038b0 0x7fae84103d30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/3302563939 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae841026b0 0x7fae84102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 -- 10.20.193.121:0/3302563939 >> 10.20.193.121:0/3302563939 conn(0x7fae840fde60 msgr2=0x7fae84100280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:28.169 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 -- 10.20.193.121:0/3302563939 shutdown_connections 2026-02-13T10:39:28.170 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 -- 10.20.193.121:0/3302563939 wait complete. 2026-02-13T10:39:28.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:27 trial121 ceph-mon[19159]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:28.170 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:27 trial121 ceph-mon[19159]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:28.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 Processor -- start 2026-02-13T10:39:28.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.167+0000 7fae8c0fc640 1 -- start start 2026-02-13T10:39:28.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae8c0fc640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841026b0 0x7fae8419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:28.172 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae8c0fc640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae841038b0 0x7fae8419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae8c0fc640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fae8410b380 con 0x7fae841026b0 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae8c0fc640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fae8410b200 con 0x7fae841038b0 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae89e71640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841026b0 0x7fae8419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae89e71640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841026b0 0x7fae8419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56848/0 (socket says 10.20.193.121:56848) 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae89e71640 1 -- 10.20.193.121:0/2427792968 learned_addr learned my addr 10.20.193.121:0/2427792968 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae89e71640 1 -- 10.20.193.121:0/2427792968 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae841038b0 msgr2=0x7fae8419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae89e71640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae841038b0 0x7fae8419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae89e71640 1 -- 10.20.193.121:0/2427792968 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8419cef0 con 0x7fae841026b0 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae89e71640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841026b0 0x7fae8419c3c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae7400ca30 tx=0x7fae7400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae72ffd640 1 -- 10.20.193.121:0/2427792968 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fae74013070 con 0x7fae841026b0 2026-02-13T10:39:28.173 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841a1960 con 0x7fae841026b0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae72ffd640 1 -- 10.20.193.121:0/2427792968 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fae74007dc0 con 0x7fae841026b0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae72ffd640 1 -- 10.20.193.121:0/2427792968 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fae7400f450 con 0x7fae841026b0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.168+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841a1f40 con 0x7fae841026b0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.169+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae84102ab0 con 0x7fae841026b0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.169+0000 7fae72ffd640 1 -- 10.20.193.121:0/2427792968 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fae74007930 con 0x7fae841026b0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.170+0000 7fae72ffd640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae60077550 0x7fae60079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.170+0000 7fae72ffd640 1 -- 10.20.193.121:0/2427792968 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7fae74099940 con 0x7fae841026b0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.170+0000 7fae89670640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae60077550 0x7fae60079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:28.174 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.170+0000 7fae89670640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae60077550 0x7fae60079a10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae780097c0 tx=0x7fae78005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:28.175 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.172+0000 7fae72ffd640 1 -- 10.20.193.121:0/2427792968 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fae74066dd0 con 0x7fae841026b0 2026-02-13T10:39:28.260 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.258+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fae840630c0 con 0x7fae60077550 2026-02-13T10:39:28.261 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.260+0000 7fae72ffd640 1 -- 10.20.193.121:0/2427792968 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+650 (secure 0 0 0) 0x7fae840630c0 con 0x7fae60077550 2026-02-13T10:39:28.261 INFO:teuthology.orchestra.run.trial121.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2026-02-13T10:39:28.262 INFO:teuthology.orchestra.run.trial121.stdout:trial121 /dev/nvme1n1 ssd MTFDKCC1T9TGP-1BK1JABYY_092510AB6C1D 1788G No 52s ago Has a FileSystem, LVM detected 2026-02-13T10:39:28.262 INFO:teuthology.orchestra.run.trial121.stdout:trial121 /dev/nvme2n1 ssd MTFDKCC1T9TGP-1BK1JABYY_012510B0043D 1788G No 52s ago Has a FileSystem, LVM detected 2026-02-13T10:39:28.262 INFO:teuthology.orchestra.run.trial121.stdout:trial163 /dev/nvme0n1 ssd MTFDKCC1T9TGP-1BK1JABYY_012510B003F5 1788G No 19s ago Has a FileSystem, LVM detected 2026-02-13T10:39:28.262 INFO:teuthology.orchestra.run.trial121.stdout:trial163 /dev/nvme2n1 ssd MTFDKCC1T9TGP-1BK1JABYY_012510B003F6 1788G No 19s ago Has a FileSystem, LVM detected 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae60077550 msgr2=0x7fae60079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae60077550 0x7fae60079a10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae780097c0 tx=0x7fae78005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841026b0 msgr2=0x7fae8419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841026b0 0x7fae8419c3c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae7400ca30 tx=0x7fae7400cf00 comp rx=0 tx=0).stop 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 shutdown_connections 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fae60077550 0x7fae60079a10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fae841038b0 0x7fae8419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.263 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 --2- 10.20.193.121:0/2427792968 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fae841026b0 0x7fae8419c3c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.264 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 >> 10.20.193.121:0/2427792968 conn(0x7fae840fde60 msgr2=0x7fae84075670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:28.264 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 shutdown_connections 2026-02-13T10:39:28.264 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.261+0000 7fae8c0fc640 1 -- 10.20.193.121:0/2427792968 wait complete. 2026-02-13T10:39:28.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:27 trial163 ceph-mon[24184]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:28.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:27 trial163 ceph-mon[24184]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:28.299 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-13T10:39:28.315 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:39:28.315 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph fs volume create cephfs --placement=4' 2026-02-13T10:39:28.601 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:28.711 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3558723586 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38105b20 msgr2=0x7f4d38105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3558723586 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38105b20 0x7f4d38105fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d200099b0 tx=0x7f4d2002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3558723586 shutdown_connections 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3558723586 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38105b20 0x7f4d38105fa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3558723586 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4d38104920 0x7f4d38104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3558723586 >> 10.20.193.121:0/3558723586 conn(0x7f4d381000b0 msgr2=0x7f4d381024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3558723586 shutdown_connections 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3558723586 wait complete. 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 Processor -- start 2026-02-13T10:39:28.712 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.708+0000 7f4d3d4ea640 1 -- start start 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d3d4ea640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38104920 0x7f4d3819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d3d4ea640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4d38105b20 0x7f4d3819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d3d4ea640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810d5f0 con 0x7f4d38104920 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d3d4ea640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810d470 con 0x7f4d38105b20 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d36ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38104920 0x7f4d3819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d36ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38104920 0x7f4d3819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56856/0 (socket says 10.20.193.121:56856) 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d36ffd640 1 -- 10.20.193.121:0/3201759598 learned_addr learned my addr 10.20.193.121:0/3201759598 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d36ffd640 1 -- 10.20.193.121:0/3201759598 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4d38105b20 msgr2=0x7f4d3819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d36ffd640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4d38105b20 0x7f4d3819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d36ffd640 1 -- 10.20.193.121:0/3201759598 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3819cff0 con 0x7f4d38104920 2026-02-13T10:39:28.713 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d36ffd640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38104920 0x7f4d3819c430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d900 tx=0x7f4d2c00ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:28.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d17fff640 1 -- 10.20.193.121:0/3201759598 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4d2c014070 con 0x7f4d38104920 2026-02-13T10:39:28.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d17fff640 1 -- 10.20.193.121:0/3201759598 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f4d2c0044a0 con 0x7f4d38104920 2026-02-13T10:39:28.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d381a1a30 con 0x7f4d38104920 2026-02-13T10:39:28.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d17fff640 1 -- 10.20.193.121:0/3201759598 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4d2c002d40 con 0x7f4d38104920 2026-02-13T10:39:28.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d381a2040 con 0x7f4d38104920 2026-02-13T10:39:28.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.709+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d38104d20 con 0x7f4d38104920 2026-02-13T10:39:28.714 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.710+0000 7f4d17fff640 1 -- 10.20.193.121:0/3201759598 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f4d2c00b840 con 0x7f4d38104920 2026-02-13T10:39:28.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.711+0000 7f4d17fff640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4d080775a0 0x7f4d08079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:28.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.711+0000 7f4d367fc640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4d080775a0 0x7f4d08079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:28.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.711+0000 7f4d17fff640 1 -- 10.20.193.121:0/3201759598 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4629+0+0 (secure 0 0 0) 0x7f4d2c099ad0 con 0x7f4d38104920 2026-02-13T10:39:28.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.711+0000 7f4d367fc640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4d080775a0 0x7f4d08079a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4d200097c0 tx=0x7f4d20005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:28.715 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.713+0000 7f4d17fff640 1 -- 10.20.193.121:0/3201759598 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4d2c099e20 con 0x7f4d38104920 2026-02-13T10:39:28.806 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:28.804+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 --> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) -- 0x7f4d380630c0 con 0x7f4d080775a0 2026-02-13T10:39:29.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:28 trial163 ceph-mon[24184]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:29.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:28 trial163 ceph-mon[24184]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:29.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:28 trial163 ceph-mon[24184]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:29.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:28 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2026-02-13T10:39:29.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:28 trial121 ceph-mon[19159]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:29.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:28 trial121 ceph-mon[19159]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:29.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:28 trial121 ceph-mon[19159]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:29.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:28 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2026-02-13T10:39:29.908 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.906+0000 7f4d17fff640 1 -- 10.20.193.121:0/3201759598 <== mgr.14203 v2:10.20.193.121:6800/2058604417 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d380630c0 con 0x7f4d080775a0 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.908+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4d080775a0 msgr2=0x7f4d08079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.908+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4d080775a0 0x7f4d08079a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4d200097c0 tx=0x7f4d20005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.908+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38104920 msgr2=0x7f4d3819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.908+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38104920 0x7f4d3819c430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d900 tx=0x7f4d2c00ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.908+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 shutdown_connections 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.908+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4d080775a0 0x7f4d08079a60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.909+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4d38105b20 0x7f4d3819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:29.911 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.909+0000 7f4d3d4ea640 1 --2- 10.20.193.121:0/3201759598 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4d38104920 0x7f4d3819c430 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:29.912 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.909+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 >> 10.20.193.121:0/3201759598 conn(0x7f4d381000b0 msgr2=0x7f4d381091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:29.912 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.909+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 shutdown_connections 2026-02-13T10:39:29.912 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:29.909+0000 7f4d3d4ea640 1 -- 10.20.193.121:0/3201759598 wait complete. 2026-02-13T10:39:29.947 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph fs dump' 2026-02-13T10:39:30.149 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:29 trial121 ceph-mon[19159]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:30.149 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:29 trial121 ceph-mon[19159]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:30.149 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:29 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2026-02-13T10:39:30.149 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:29 trial121 ceph-mon[19159]: osdmap e38: 6 total, 6 up, 6 in 2026-02-13T10:39:30.149 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:29 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2026-02-13T10:39:30.149 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:29 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121[19155]: 2026-02-13T10:39:29.899+0000 7faa4871a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2026-02-13T10:39:30.241 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:30.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:29 trial163 ceph-mon[24184]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:30.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:29 trial163 ceph-mon[24184]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T10:39:30.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:29 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2026-02-13T10:39:30.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:29 trial163 ceph-mon[24184]: osdmap e38: 6 total, 6 up, 6 in 2026-02-13T10:39:30.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:29 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2026-02-13T10:39:30.392 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.387+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/360607965 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 msgr2=0x7f7cb410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.387+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/360607965 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb410a040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80098e0 tx=0x7f7ca802f1d0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.388+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/360607965 shutdown_connections 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.388+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/360607965 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb410a040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.388+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/360607965 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7cb406b900 0x7f7cb4107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.388+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/360607965 >> 10.20.193.121:0/360607965 conn(0x7f7cb40fcd20 msgr2=0x7f7cb40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.389+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/360607965 shutdown_connections 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.389+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/360607965 wait complete. 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.390+0000 7f7cbbbaf640 1 Processor -- start 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.390+0000 7f7cbbbaf640 1 -- start start 2026-02-13T10:39:30.393 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.390+0000 7f7cbbbaf640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7cb406b900 0x7f7cb419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:30.394 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.390+0000 7f7cbbbaf640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:30.394 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.390+0000 7f7cbbbaf640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb410d640 con 0x7f7cb4107c50 2026-02-13T10:39:30.394 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.390+0000 7f7cbbbaf640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb410d4c0 con 0x7f7cb406b900 2026-02-13T10:39:30.394 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.391+0000 7f7cb9123640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:30.394 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.391+0000 7f7cb9123640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56868/0 (socket says 10.20.193.121:56868) 2026-02-13T10:39:30.394 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.391+0000 7f7cb9123640 1 -- 10.20.193.121:0/2198737738 learned_addr learned my addr 10.20.193.121:0/2198737738 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:30.395 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.391+0000 7f7cb9123640 1 -- 10.20.193.121:0/2198737738 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7cb406b900 msgr2=0x7f7cb419c410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T10:39:30.395 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.391+0000 7f7cb9123640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7cb406b900 0x7f7cb419c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.395 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.391+0000 7f7cb9123640 1 -- 10.20.193.121:0/2198737738 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb419cfd0 con 0x7f7cb4107c50 2026-02-13T10:39:30.395 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.391+0000 7f7cb9123640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb419c950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8002410 tx=0x7f7ca802ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:30.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.392+0000 7f7ca2ffd640 1 -- 10.20.193.121:0/2198737738 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7ca8046070 con 0x7f7cb4107c50 2026-02-13T10:39:30.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.392+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb41a19b0 con 0x7f7cb4107c50 2026-02-13T10:39:30.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.392+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb41a1fc0 con 0x7f7cb4107c50 2026-02-13T10:39:30.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.392+0000 7f7ca2ffd640 1 -- 10.20.193.121:0/2198737738 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f7ca80044a0 con 0x7f7cb4107c50 2026-02-13T10:39:30.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.392+0000 7f7ca2ffd640 1 -- 10.20.193.121:0/2198737738 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7ca8041750 con 0x7f7cb4107c50 2026-02-13T10:39:30.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.394+0000 7f7ca2ffd640 1 -- 10.20.193.121:0/2198737738 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f7ca8038510 con 0x7f7cb4107c50 2026-02-13T10:39:30.396 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.394+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7c005390 con 0x7f7cb4107c50 2026-02-13T10:39:30.397 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.394+0000 7f7ca2ffd640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c900775a0 0x7f7c90079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:30.397 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.394+0000 7f7ca2ffd640 1 -- 10.20.193.121:0/2198737738 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5459+0+0 (secure 0 0 0) 0x7f7ca803e070 con 0x7f7cb4107c50 2026-02-13T10:39:30.397 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.394+0000 7f7cb9924640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c900775a0 0x7f7c90079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:30.397 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.394+0000 7f7cb9924640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c900775a0 0x7f7c90079a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009b70 tx=0x7f7ca4009340 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:30.398 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.397+0000 7f7ca2ffd640 1 -- 10.20.193.121:0/2198737738 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7ca808b000 con 0x7f7cb4107c50 2026-02-13T10:39:30.501 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.499+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7c7c005e90 con 0x7f7cb4107c50 2026-02-13T10:39:30.502 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.499+0000 7f7ca2ffd640 1 -- 10.20.193.121:0/2198737738 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) ==== 75+0+1272 (secure 0 0 0) 0x7f7ca808feb0 con 0x7f7cb4107c50 2026-02-13T10:39:30.502 INFO:teuthology.orchestra.run.trial121.stderr:dumped fsmap epoch 2 2026-02-13T10:39:30.502 INFO:teuthology.orchestra.run.trial121.stdout:e2 2026-02-13T10:39:30.502 INFO:teuthology.orchestra.run.trial121.stdout:btime 2026-02-13T10:39:29:899982+0000 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:enable_multiple, ever_enabled_multiple: 1,1 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:legacy client fscid: 1 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:Filesystem 'cephfs' (1) 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:fs_name cephfs 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:epoch 2 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:created 2026-02-13T10:39:29.899884+0000 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:modified 2026-02-13T10:39:29.899884+0000 2026-02-13T10:39:30.503 INFO:teuthology.orchestra.run.trial121.stdout:tableserver 0 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:root 0 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:session_timeout 60 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:session_autoclose 300 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:max_file_size 1099511627776 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:max_xattr_size 65536 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:required_client_features {} 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:last_failure 0 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:last_failure_osd_epoch 0 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:max_mds 1 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:in 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:up {} 2026-02-13T10:39:30.504 INFO:teuthology.orchestra.run.trial121.stdout:failed 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:damaged 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:stopped 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:data_pools [3] 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:metadata_pool 2 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:inline_data disabled 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:balancer 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:bal_rank_mask -1 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:standby_count_wanted 0 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout:qdb_cluster leader: 0 members: 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:30.505 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:30.506 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c900775a0 msgr2=0x7f7c90079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:30.506 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c900775a0 0x7f7c90079a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009b70 tx=0x7f7ca4009340 comp rx=0 tx=0).stop 2026-02-13T10:39:30.506 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 msgr2=0x7f7cb419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb419c950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8002410 tx=0x7f7ca802ff90 comp rx=0 tx=0).stop 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 shutdown_connections 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7c900775a0 0x7f7c90079a60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7cb4107c50 0x7f7cb419c950 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 --2- 10.20.193.121:0/2198737738 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7cb406b900 0x7f7cb419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 >> 10.20.193.121:0/2198737738 conn(0x7f7cb40fcd20 msgr2=0x7f7cb40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 shutdown_connections 2026-02-13T10:39:30.507 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.501+0000 7f7cbbbaf640 1 -- 10.20.193.121:0/2198737738 wait complete. 2026-02-13T10:39:30.544 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-13T10:39:30.559 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:39:30.560 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph fs set cephfs max_mds 1' 2026-02-13T10:39:30.856 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:30.964 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- 10.20.193.121:0/4010739837 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 msgr2=0x7f4ee4105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:30.964 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/4010739837 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee4105fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0099b0 tx=0x7f4ecc02f220 comp rx=0 tx=0).stop 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- 10.20.193.121:0/4010739837 shutdown_connections 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/4010739837 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee4105fa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/4010739837 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4ee4104920 0x7f4ee4104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- 10.20.193.121:0/4010739837 >> 10.20.193.121:0/4010739837 conn(0x7f4ee41000b0 msgr2=0x7f4ee41024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- 10.20.193.121:0/4010739837 shutdown_connections 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- 10.20.193.121:0/4010739837 wait complete. 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 Processor -- start 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- start start 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4ee4104920 0x7f4ee419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:30.965 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee410d5f0 con 0x7f4ee4105b20 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee98b2640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee410d470 con 0x7f4ee4104920 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee27fc640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee27fc640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56906/0 (socket says 10.20.193.121:56906) 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.961+0000 7f4ee27fc640 1 -- 10.20.193.121:0/1731634037 learned_addr learned my addr 10.20.193.121:0/1731634037 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee2ffd640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4ee4104920 0x7f4ee419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee27fc640 1 -- 10.20.193.121:0/1731634037 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4ee4104920 msgr2=0x7f4ee419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee27fc640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4ee4104920 0x7f4ee419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee27fc640 1 -- 10.20.193.121:0/1731634037 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee419cff0 con 0x7f4ee4105b20 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee27fc640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee419c970 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc002410 tx=0x7f4ecc02fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee88b0640 1 -- 10.20.193.121:0/1731634037 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4ecc004260 con 0x7f4ee4105b20 2026-02-13T10:39:30.966 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee88b0640 1 -- 10.20.193.121:0/1731634037 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f4ecc004400 con 0x7f4ee4105b20 2026-02-13T10:39:30.967 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41a19d0 con 0x7f4ee4105b20 2026-02-13T10:39:30.967 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee88b0640 1 -- 10.20.193.121:0/1731634037 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f4ecc041960 con 0x7f4ee4105b20 2026-02-13T10:39:30.967 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a1fe0 con 0x7f4ee4105b20 2026-02-13T10:39:30.967 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.962+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee4104d20 con 0x7f4ee4105b20 2026-02-13T10:39:30.968 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.963+0000 7f4ee88b0640 1 -- 10.20.193.121:0/1731634037 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f4ecc041c60 con 0x7f4ee4105b20 2026-02-13T10:39:30.968 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.964+0000 7f4ee88b0640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4eb8077550 0x7f4eb8079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:30.968 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.964+0000 7f4ee2ffd640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4eb8077550 0x7f4eb8079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:30.968 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.964+0000 7f4ee88b0640 1 -- 10.20.193.121:0/1731634037 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5459+0+0 (secure 0 0 0) 0x7f4ecc03e070 con 0x7f4ee4105b20 2026-02-13T10:39:30.968 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.964+0000 7f4ee2ffd640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4eb8077550 0x7f4eb8079a10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8005fd0 tx=0x7f4ed8004380 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:30.968 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:30.966+0000 7f4ee88b0640 1 -- 10.20.193.121:0/1731634037 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4ecc08b8e0 con 0x7f4ee4105b20 2026-02-13T10:39:31.066 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.065+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f4ee4196000 con 0x7f4ee4105b20 2026-02-13T10:39:31.113 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: pgmap v78: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:31.113 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: osdmap e39: 6 total, 6 up, 6 in 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: osdmap e40: 6 total, 6 up, 6 in 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: fsmap cephfs:0 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: Saving service mds.cephfs spec with placement count:4 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:31.114 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.sfinrg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.sfinrg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2198737738' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.mabprx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:31.115 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.mabprx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:31.116 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:30 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:31.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: pgmap v78: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:31.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2026-02-13T10:39:31.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: osdmap e39: 6 total, 6 up, 6 in 2026-02-13T10:39:31.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: osdmap e40: 6 total, 6 up, 6 in 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: fsmap cephfs:0 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: Saving service mds.cephfs spec with placement count:4 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:31.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.sfinrg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.sfinrg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2198737738' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.mabprx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.mabprx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:31.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:30 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:31.574 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.573+0000 7f4ee88b0640 1 -- 10.20.193.121:0/1731634037 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v3) ==== 105+0+0 (secure 0 0 0) 0x7f4ecc090790 con 0x7f4ee4105b20 2026-02-13T10:39:31.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4eb8077550 msgr2=0x7f4eb8079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:31.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4eb8077550 0x7f4eb8079a10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8005fd0 tx=0x7f4ed8004380 comp rx=0 tx=0).stop 2026-02-13T10:39:31.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 msgr2=0x7f4ee419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:31.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee419c970 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc002410 tx=0x7f4ecc02fc90 comp rx=0 tx=0).stop 2026-02-13T10:39:31.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 shutdown_connections 2026-02-13T10:39:31.577 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f4eb8077550 0x7f4eb8079a10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:31.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f4ee4105b20 0x7f4ee419c970 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:31.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 --2- 10.20.193.121:0/1731634037 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f4ee4104920 0x7f4ee419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:31.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 >> 10.20.193.121:0/1731634037 conn(0x7f4ee41000b0 msgr2=0x7f4ee41091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:31.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 shutdown_connections 2026-02-13T10:39:31.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:31.575+0000 7f4ee98b2640 1 -- 10.20.193.121:0/1731634037 wait complete. 2026-02-13T10:39:31.617 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-13T10:39:31.633 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:39:31.634 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2026-02-13T10:39:32.064 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:32.180 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: Deploying daemon mds.cephfs.trial121.sfinrg on trial121 2026-02-13T10:39:32.180 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: Deploying daemon mds.cephfs.trial163.mabprx on trial163 2026-02-13T10:39:32.180 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: osdmap e41: 6 total, 6 up, 6 in 2026-02-13T10:39:32.180 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1731634037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2026-02-13T10:39:32.180 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.blaxtt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.blaxtt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1731634037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:boot 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.163:6824/3424379208,v1:10.20.193.163:6825/3424379208] up:boot 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: daemon mds.cephfs.trial163.mabprx assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2026-02-13T10:39:32.181 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: Cluster is now healthy 2026-02-13T10:39:32.182 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: fsmap cephfs:0 2 up:standby 2026-02-13T10:39:32.182 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial121.sfinrg"}]: dispatch 2026-02-13T10:39:32.182 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial163.mabprx"}]: dispatch 2026-02-13T10:39:32.182 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:creating} 1 up:standby 2026-02-13T10:39:32.182 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:31 trial121 ceph-mon[19159]: daemon mds.cephfs.trial163.mabprx is now active in filesystem cephfs as rank 0 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 -- 10.20.193.121:0/1656907212 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 msgr2=0x7f9d54105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1656907212 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d54105fc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9d440099b0 tx=0x7f9d4402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 -- 10.20.193.121:0/1656907212 shutdown_connections 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1656907212 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d54105fc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1656907212 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9d54104940 0x7f9d54104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 -- 10.20.193.121:0/1656907212 >> 10.20.193.121:0/1656907212 conn(0x7f9d541000f0 msgr2=0x7f9d54102510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 -- 10.20.193.121:0/1656907212 shutdown_connections 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.178+0000 7f9d59b62640 1 -- 10.20.193.121:0/1656907212 wait complete. 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 Processor -- start 2026-02-13T10:39:32.185 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 -- start start 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9d54104940 0x7f9d5407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d5407b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5410d630 con 0x7f9d54105b40 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5410d4b0 con 0x7f9d54104940 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d52ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d5407b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d52ffd640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d5407b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56942/0 (socket says 10.20.193.121:56942) 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d52ffd640 1 -- 10.20.193.121:0/1975695543 learned_addr learned my addr 10.20.193.121:0/1975695543 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d537fe640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9d54104940 0x7f9d5407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d52ffd640 1 -- 10.20.193.121:0/1975695543 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9d54104940 msgr2=0x7f9d5407ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d52ffd640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9d54104940 0x7f9d5407ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d52ffd640 1 -- 10.20.193.121:0/1975695543 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d54077900 con 0x7f9d54105b40 2026-02-13T10:39:32.186 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d52ffd640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d5407b370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d4402fc90 tx=0x7f9d4402fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d50ff9640 1 -- 10.20.193.121:0/1975695543 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9d44004420 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d50ff9640 1 -- 10.20.193.121:0/1975695543 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f9d44038c00 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d50ff9640 1 -- 10.20.193.121:0/1975695543 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f9d44041950 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d54077b90 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.179+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d540780f0 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.180+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d54104d40 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.181+0000 7f9d50ff9640 1 -- 10.20.193.121:0/1975695543 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f9d44041c00 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.181+0000 7f9d50ff9640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9d280775a0 0x7f9d28079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.181+0000 7f9d50ff9640 1 -- 10.20.193.121:0/1975695543 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5459+0+0 (secure 0 0 0) 0x7f9d4403e070 con 0x7f9d54105b40 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.181+0000 7f9d537fe640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9d280775a0 0x7f9d28079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.181+0000 7f9d537fe640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9d280775a0 0x7f9d28079a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d541059a0 tx=0x7f9d40009290 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:32.187 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.183+0000 7f9d50ff9640 1 -- 10.20.193.121:0/1975695543 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9d44049340 con 0x7f9d54105b40 2026-02-13T10:39:32.284 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.283+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9d541963a0 con 0x7f9d54105b40 2026-02-13T10:39:32.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: Deploying daemon mds.cephfs.trial121.sfinrg on trial121 2026-02-13T10:39:32.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: Deploying daemon mds.cephfs.trial163.mabprx on trial163 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: osdmap e41: 6 total, 6 up, 6 in 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1731634037' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.blaxtt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial121.blaxtt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1731634037' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:boot 2026-02-13T10:39:32.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.163:6824/3424379208,v1:10.20.193.163:6825/3424379208] up:boot 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: daemon mds.cephfs.trial163.mabprx assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: Cluster is now healthy 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: fsmap cephfs:0 2 up:standby 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial121.sfinrg"}]: dispatch 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial163.mabprx"}]: dispatch 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:creating} 1 up:standby 2026-02-13T10:39:32.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:31 trial163 ceph-mon[24184]: daemon mds.cephfs.trial163.mabprx is now active in filesystem cephfs as rank 0 2026-02-13T10:39:32.578 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.576+0000 7f9d50ff9640 1 -- 10.20.193.121:0/1975695543 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v5) ==== 122+0+0 (secure 0 0 0) 0x7f9d4408b8e0 con 0x7f9d54105b40 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9d280775a0 msgr2=0x7f9d28079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9d280775a0 0x7f9d28079a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d541059a0 tx=0x7f9d40009290 comp rx=0 tx=0).stop 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 msgr2=0x7f9d5407b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d5407b370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d4402fc90 tx=0x7f9d4402fcc0 comp rx=0 tx=0).stop 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 shutdown_connections 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f9d280775a0 0x7f9d28079a60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f9d54105b40 0x7f9d5407b370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 --2- 10.20.193.121:0/1975695543 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f9d54104940 0x7f9d5407ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 >> 10.20.193.121:0/1975695543 conn(0x7f9d541000f0 msgr2=0x7f9d54109190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:32.580 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 shutdown_connections 2026-02-13T10:39:32.581 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:32.578+0000 7f9d59b62640 1 -- 10.20.193.121:0/1975695543 wait complete. 2026-02-13T10:39:32.618 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-13T10:39:32.634 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:39:32.635 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2026-02-13T10:39:32.929 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:33.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.042+0000 7f2c20e07640 1 -- 10.20.193.121:0/1704213357 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1038b0 msgr2=0x7f2c1c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:33.046 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.042+0000 7f2c20e07640 1 --2- 10.20.193.121:0/1704213357 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1038b0 0x7f2c1c103d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2c040099b0 tx=0x7f2c0402f220 comp rx=0 tx=0).stop 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.042+0000 7f2c20e07640 1 -- 10.20.193.121:0/1704213357 shutdown_connections 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.042+0000 7f2c20e07640 1 --2- 10.20.193.121:0/1704213357 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1038b0 0x7f2c1c103d30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.042+0000 7f2c20e07640 1 --2- 10.20.193.121:0/1704213357 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2c1c1026b0 0x7f2c1c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.042+0000 7f2c20e07640 1 -- 10.20.193.121:0/1704213357 >> 10.20.193.121:0/1704213357 conn(0x7f2c1c0fde60 msgr2=0x7f2c1c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.042+0000 7f2c20e07640 1 -- 10.20.193.121:0/1704213357 shutdown_connections 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 -- 10.20.193.121:0/1704213357 wait complete. 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 Processor -- start 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 -- start start 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1026b0 0x7f2c1c076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2c1c1038b0 0x7f2c1c075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:33.047 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c10b380 con 0x7f2c1c1026b0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c10b200 con 0x7f2c1c1038b0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c1a575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1026b0 0x7f2c1c076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c1a575640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1026b0 0x7f2c1c076da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56970/0 (socket says 10.20.193.121:56970) 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c1a575640 1 -- 10.20.193.121:0/416297310 learned_addr learned my addr 10.20.193.121:0/416297310 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c19d74640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2c1c1038b0 0x7f2c1c075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c1a575640 1 -- 10.20.193.121:0/416297310 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2c1c1038b0 msgr2=0x7f2c1c075460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c1a575640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2c1c1038b0 0x7f2c1c075460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c1a575640 1 -- 10.20.193.121:0/416297310 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c0759a0 con 0x7f2c1c1026b0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c1a575640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1026b0 0x7f2c1c076da0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000ca30 tx=0x7f2c1000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c037fe640 1 -- 10.20.193.121:0/416297310 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f2c10013070 con 0x7f2c1c1026b0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c037fe640 1 -- 10.20.193.121:0/416297310 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f2c10007dc0 con 0x7f2c1c1026b0 2026-02-13T10:39:33.048 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c075c90 con 0x7f2c1c1026b0 2026-02-13T10:39:33.049 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c037fe640 1 -- 10.20.193.121:0/416297310 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f2c1000f450 con 0x7f2c1c1026b0 2026-02-13T10:39:33.049 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.043+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c0762a0 con 0x7f2c1c1026b0 2026-02-13T10:39:33.049 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.044+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c102ab0 con 0x7f2c1c1026b0 2026-02-13T10:39:33.049 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.045+0000 7f2c037fe640 1 -- 10.20.193.121:0/416297310 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f2c10007930 con 0x7f2c1c1026b0 2026-02-13T10:39:33.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.045+0000 7f2c037fe640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2bf40775a0 0x7f2bf4079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:33.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.045+0000 7f2c19d74640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2bf40775a0 0x7f2bf4079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:33.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.045+0000 7f2c037fe640 1 -- 10.20.193.121:0/416297310 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5459+0+0 (secure 0 0 0) 0x7f2c10099840 con 0x7f2c1c1026b0 2026-02-13T10:39:33.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.046+0000 7f2c19d74640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2bf40775a0 0x7f2bf4079a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c076b20 tx=0x7f2c0403a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:33.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.048+0000 7f2c037fe640 1 -- 10.20.193.121:0/416297310 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2c10066990 con 0x7f2c1c1026b0 2026-02-13T10:39:33.155 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.154+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2c1c1a2a50 con 0x7f2c1c1026b0 2026-02-13T10:39:33.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: Deploying daemon mds.cephfs.trial121.blaxtt on trial121 2026-02-13T10:39:33.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: pgmap v82: 65 pgs: 3 creating+peering, 58 unknown, 4 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:33.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: osdmap e42: 6 total, 6 up, 6 in 2026-02-13T10:39:33.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.bxtdjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.bxtdjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1975695543' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1975695543' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.163:6824/3424379208,v1:10.20.193.163:6825/3424379208] up:active 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.121:6828/2967641829,v1:10.20.193.121:6829/2967641829] up:boot 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:active} 2 up:standby 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial121.blaxtt"}]: dispatch 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:active} 2 up:standby 2026-02-13T10:39:33.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:32 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: Deploying daemon mds.cephfs.trial121.blaxtt on trial121 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: pgmap v82: 65 pgs: 3 creating+peering, 58 unknown, 4 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: osdmap e42: 6 total, 6 up, 6 in 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.bxtdjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.trial163.bxtdjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1975695543' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2026-02-13T10:39:33.245 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1975695543' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.163:6824/3424379208,v1:10.20.193.163:6825/3424379208] up:active 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.121:6828/2967641829,v1:10.20.193.121:6829/2967641829] up:boot 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:active} 2 up:standby 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial121.blaxtt"}]: dispatch 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:active} 2 up:standby 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:33.246 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:32 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:33.582 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.580+0000 7f2c037fe640 1 -- 10.20.193.121:0/416297310 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v7) ==== 168+0+0 (secure 0 0 0) 0x7f2c1006b840 con 0x7f2c1c1026b0 2026-02-13T10:39:33.582 INFO:teuthology.orchestra.run.trial121.stderr:inline data enabled 2026-02-13T10:39:33.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2bf40775a0 msgr2=0x7f2bf4079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:33.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2bf40775a0 0x7f2bf4079a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c076b20 tx=0x7f2c0403a040 comp rx=0 tx=0).stop 2026-02-13T10:39:33.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1026b0 msgr2=0x7f2c1c076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:33.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1026b0 0x7f2c1c076da0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000ca30 tx=0x7f2c1000cf00 comp rx=0 tx=0).stop 2026-02-13T10:39:33.584 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 shutdown_connections 2026-02-13T10:39:33.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f2bf40775a0 0x7f2bf4079a60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:33.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f2c1c1038b0 0x7f2c1c075460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:33.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 --2- 10.20.193.121:0/416297310 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f2c1c1026b0 0x7f2c1c076da0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:33.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 >> 10.20.193.121:0/416297310 conn(0x7f2c1c0fde60 msgr2=0x7f2c1c100230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:33.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 shutdown_connections 2026-02-13T10:39:33.585 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:33.582+0000 7f2c20e07640 1 -- 10.20.193.121:0/416297310 wait complete. 2026-02-13T10:39:33.624 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-13T10:39:33.640 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:39:33.640 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph fs dump' 2026-02-13T10:39:33.935 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:34.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: Deploying daemon mds.cephfs.trial163.bxtdjl on trial163 2026-02-13T10:39:34.040 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/416297310' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/416297310' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.163:6826/2686239026,v1:10.20.193.163:6827/2686239026] up:boot 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:active} 3 up:standby 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial163.bxtdjl"}]: dispatch 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.041 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:34.042 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:34.042 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:33 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1722389113 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f006b900 msgr2=0x7fe0f0107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1722389113 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f006b900 0x7fe0f0107710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00098e0 tx=0x7fe0e002f1d0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1722389113 shutdown_connections 2026-02-13T10:39:34.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1722389113 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe0f0107c50 0x7fe0f010a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.050 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1722389113 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f006b900 0x7fe0f0107710 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1722389113 >> 10.20.193.121:0/1722389113 conn(0x7fe0f00fcd20 msgr2=0x7fe0f00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1722389113 shutdown_connections 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1722389113 wait complete. 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 Processor -- start 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- start start 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe0f006b900 0x7fe0f019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f0107c50 0x7fe0f019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010d620 con 0x7fe0f0107c50 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.046+0000 7fe0f6d5b640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010d4a0 con 0x7fe0f006b900 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e7fff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f0107c50 0x7fe0f019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e7fff640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f0107c50 0x7fe0f019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:56992/0 (socket says 10.20.193.121:56992) 2026-02-13T10:39:34.051 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e7fff640 1 -- 10.20.193.121:0/1193641152 learned_addr learned my addr 10.20.193.121:0/1193641152 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0f4ad0640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe0f006b900 0x7fe0f019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e7fff640 1 -- 10.20.193.121:0/1193641152 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe0f006b900 msgr2=0x7fe0f019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e7fff640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe0f006b900 0x7fe0f019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e7fff640 1 -- 10.20.193.121:0/1193641152 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f019cfb0 con 0x7fe0f0107c50 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e7fff640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f0107c50 0x7fe0f019c930 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8002fd0 tx=0x7fe0d800da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e5ffb640 1 -- 10.20.193.121:0/1193641152 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fe0d8014070 con 0x7fe0f0107c50 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e5ffb640 1 -- 10.20.193.121:0/1193641152 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fe0d8009d80 con 0x7fe0f0107c50 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01a19f0 con 0x7fe0f0107c50 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0e5ffb640 1 -- 10.20.193.121:0/1193641152 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fe0d800f3c0 con 0x7fe0f0107c50 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01a2000 con 0x7fe0f0107c50 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.047+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f0107710 con 0x7fe0f0107c50 2026-02-13T10:39:34.052 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.048+0000 7fe0e5ffb640 1 -- 10.20.193.121:0/1193641152 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fe0d8010460 con 0x7fe0f0107c50 2026-02-13T10:39:34.053 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.048+0000 7fe0e5ffb640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0cc0775a0 0x7fe0cc079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:34.053 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.048+0000 7fe0f4ad0640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0cc0775a0 0x7fe0cc079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:34.053 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.049+0000 7fe0e5ffb640 1 -- 10.20.193.121:0/1193641152 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5459+0+0 (secure 0 0 0) 0x7fe0d8099a10 con 0x7fe0f0107c50 2026-02-13T10:39:34.053 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.049+0000 7fe0f4ad0640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0cc0775a0 0x7fe0cc079a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00096f0 tx=0x7fe0e003a040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:34.053 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.051+0000 7fe0e5ffb640 1 -- 10.20.193.121:0/1193641152 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe0d8066b60 con 0x7fe0f0107c50 2026-02-13T10:39:34.158 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.156+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe0f01a29a0 con 0x7fe0f0107c50 2026-02-13T10:39:34.158 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.156+0000 7fe0e5ffb640 1 -- 10.20.193.121:0/1193641152 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 7 v7) ==== 75+0+1965 (secure 0 0 0) 0x7fe0d806ba10 con 0x7fe0f0107c50 2026-02-13T10:39:34.159 INFO:teuthology.orchestra.run.trial121.stdout:e7 2026-02-13T10:39:34.159 INFO:teuthology.orchestra.run.trial121.stdout:btime 2026-02-13T10:39:33:580098+0000 2026-02-13T10:39:34.159 INFO:teuthology.orchestra.run.trial121.stdout:enable_multiple, ever_enabled_multiple: 1,1 2026-02-13T10:39:34.159 INFO:teuthology.orchestra.run.trial121.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-13T10:39:34.159 INFO:teuthology.orchestra.run.trial121.stdout:legacy client fscid: 1 2026-02-13T10:39:34.159 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:34.159 INFO:teuthology.orchestra.run.trial121.stdout:Filesystem 'cephfs' (1) 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:fs_name cephfs 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:epoch 7 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:created 2026-02-13T10:39:29.899884+0000 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:modified 2026-02-13T10:39:33.155510+0000 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:tableserver 0 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:root 0 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:session_timeout 60 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:session_autoclose 300 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:max_file_size 1099511627776 2026-02-13T10:39:34.160 INFO:teuthology.orchestra.run.trial121.stdout:max_xattr_size 65536 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:required_client_features {} 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:last_failure 0 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:last_failure_osd_epoch 0 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:max_mds 1 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:in 0 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:up {0=24295} 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:failed 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:damaged 2026-02-13T10:39:34.161 INFO:teuthology.orchestra.run.trial121.stdout:stopped 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:data_pools [3] 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:metadata_pool 2 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:inline_data enabled 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:balancer 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:bal_rank_mask -1 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:standby_count_wanted 1 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:qdb_cluster leader: 24295 members: 24295 2026-02-13T10:39:34.162 INFO:teuthology.orchestra.run.trial121.stdout:[mds.cephfs.trial163.mabprx{0:24295} state up:active seq 2 addr [v2:10.20.193.163:6824/3424379208,v1:10.20.193.163:6825/3424379208] compat {c=[1],r=[1],i=[1fff]}] 2026-02-13T10:39:34.163 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:34.163 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:34.163 INFO:teuthology.orchestra.run.trial121.stdout:Standby daemons: 2026-02-13T10:39:34.163 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:34.163 INFO:teuthology.orchestra.run.trial121.stdout:[mds.cephfs.trial121.sfinrg{-1:14602} state up:standby seq 1 addr [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] compat {c=[1],r=[1],i=[1fff]}] 2026-02-13T10:39:34.163 INFO:teuthology.orchestra.run.trial121.stdout:[mds.cephfs.trial121.blaxtt{-1:14612} state up:standby seq 1 addr [v2:10.20.193.121:6828/2967641829,v1:10.20.193.121:6829/2967641829] compat {c=[1],r=[1],i=[1fff]}] 2026-02-13T10:39:34.163 INFO:teuthology.orchestra.run.trial121.stdout:[mds.cephfs.trial163.bxtdjl{-1:24305} state up:standby seq 1 addr [v2:10.20.193.163:6826/2686239026,v1:10.20.193.163:6827/2686239026] compat {c=[1],r=[1],i=[1fff]}] 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:dumped fsmap epoch 7 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0cc0775a0 msgr2=0x7fe0cc079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0cc0775a0 0x7fe0cc079a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00096f0 tx=0x7fe0e003a040 comp rx=0 tx=0).stop 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f0107c50 msgr2=0x7fe0f019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f0107c50 0x7fe0f019c930 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8002fd0 tx=0x7fe0d800da70 comp rx=0 tx=0).stop 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 shutdown_connections 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fe0cc0775a0 0x7fe0cc079a60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fe0f0107c50 0x7fe0f019c930 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 --2- 10.20.193.121:0/1193641152 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fe0f006b900 0x7fe0f019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 >> 10.20.193.121:0/1193641152 conn(0x7fe0f00fcd20 msgr2=0x7fe0f00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:34.165 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 shutdown_connections 2026-02-13T10:39:34.166 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.158+0000 7fe0f6d5b640 1 -- 10.20.193.121:0/1193641152 wait complete. 2026-02-13T10:39:34.166 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: Deploying daemon mds.cephfs.trial163.bxtdjl on trial163 2026-02-13T10:39:34.166 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/416297310' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2026-02-13T10:39:34.166 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.166 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.166 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2026-02-13T10:39:34.166 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/416297310' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2026-02-13T10:39:34.166 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.163:6826/2686239026,v1:10.20.193.163:6827/2686239026] up:boot 2026-02-13T10:39:34.167 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: fsmap cephfs:1 {0=cephfs.trial163.mabprx=up:active} 3 up:standby 2026-02-13T10:39:34.167 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial163.bxtdjl"}]: dispatch 2026-02-13T10:39:34.167 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.167 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.167 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:34.167 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:34.167 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:33 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:34.198 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2026-02-13T10:39:34.491 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:34.614 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- 10.20.193.121:0/3717730449 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 msgr2=0x7fd508104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.614 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 --2- 10.20.193.121:0/3717730449 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd508104b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80099b0 tx=0x7fd4f802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:34.614 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- 10.20.193.121:0/3717730449 shutdown_connections 2026-02-13T10:39:34.614 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 --2- 10.20.193.121:0/3717730449 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd508105950 0x7fd508105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.614 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 --2- 10.20.193.121:0/3717730449 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd508104b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- 10.20.193.121:0/3717730449 >> 10.20.193.121:0/3717730449 conn(0x7fd5080fff00 msgr2=0x7fd508102320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- 10.20.193.121:0/3717730449 shutdown_connections 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- 10.20.193.121:0/3717730449 wait complete. 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 Processor -- start 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- start start 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd50819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd508105950 0x7fd50819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd50810d4a0 con 0x7fd508104750 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.610+0000 7fd50df8a640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd50810d320 con 0x7fd508105950 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50cf88640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd50819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:34.615 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50cf88640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd50819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57000/0 (socket says 10.20.193.121:57000) 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50cf88640 1 -- 10.20.193.121:0/146082484 learned_addr learned my addr 10.20.193.121:0/146082484 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd4fffff640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd508105950 0x7fd50819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50cf88640 1 -- 10.20.193.121:0/146082484 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd508105950 msgr2=0x7fd50819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50cf88640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd508105950 0x7fd50819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50cf88640 1 -- 10.20.193.121:0/146082484 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50819ceb0 con 0x7fd508104750 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50cf88640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd50819c2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4f802f730 tx=0x7fd4f8038660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd4fdffb640 1 -- 10.20.193.121:0/146082484 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fd4f8046070 con 0x7fd508104750 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5081a1890 con 0x7fd508104750 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd4fdffb640 1 -- 10.20.193.121:0/146082484 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fd4f80044b0 con 0x7fd508104750 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd4fdffb640 1 -- 10.20.193.121:0/146082484 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fd4f8041a50 con 0x7fd508104750 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.611+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5081a1ea0 con 0x7fd508104750 2026-02-13T10:39:34.616 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.612+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd508104bd0 con 0x7fd508104750 2026-02-13T10:39:34.617 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.612+0000 7fd4fdffb640 1 -- 10.20.193.121:0/146082484 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fd4f8038770 con 0x7fd508104750 2026-02-13T10:39:34.618 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.613+0000 7fd4fdffb640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd4e4077550 0x7fd4e4079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:34.618 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.613+0000 7fd4fffff640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd4e4077550 0x7fd4e4079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:34.618 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.613+0000 7fd4fdffb640 1 -- 10.20.193.121:0/146082484 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5459+0+0 (secure 0 0 0) 0x7fd4f803e070 con 0x7fd508104750 2026-02-13T10:39:34.618 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.613+0000 7fd4fffff640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd4e4077550 0x7fd4e4079a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd50819d7a0 tx=0x7fd4f0008040 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:34.618 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.614+0000 7fd4fdffb640 1 -- 10.20.193.121:0/146082484 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd4f8050020 con 0x7fd508104750 2026-02-13T10:39:34.727 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.725+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5081a2130 con 0x7fd508104750 2026-02-13T10:39:34.728 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.726+0000 7fd4fdffb640 1 -- 10.20.193.121:0/146082484 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5321 (secure 0 0 0) 0x7fd4f808bde0 con 0x7fd508104750 2026-02-13T10:39:34.728 INFO:teuthology.orchestra.run.trial121.stderr:dumped fsmap epoch 7 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd4e4077550 msgr2=0x7fd4e4079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd4e4077550 0x7fd4e4079a10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd50819d7a0 tx=0x7fd4f0008040 comp rx=0 tx=0).stop 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 msgr2=0x7fd50819c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd50819c2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd4f802f730 tx=0x7fd4f8038660 comp rx=0 tx=0).stop 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 shutdown_connections 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd4e4077550 0x7fd4e4079a10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd508105950 0x7fd50819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.730 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 --2- 10.20.193.121:0/146082484 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd508104750 0x7fd50819c2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:34.731 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 >> 10.20.193.121:0/146082484 conn(0x7fd5080fff00 msgr2=0x7fd508108b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:34.731 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 shutdown_connections 2026-02-13T10:39:34.731 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:34.728+0000 7fd50df8a640 1 -- 10.20.193.121:0/146082484 wait complete. 2026-02-13T10:39:34.737 INFO:teuthology.orchestra.run.trial121.stdout:true 2026-02-13T10:39:34.770 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2026-02-13T10:39:35.065 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:35.088 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:34 trial121 ceph-mon[19159]: pgmap v84: 65 pgs: 3 creating+peering, 24 unknown, 38 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 2.5 KiB/s wr, 3 op/s 2026-02-13T10:39:35.089 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:34 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:35.089 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:34 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1193641152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2026-02-13T10:39:35.089 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:34 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:35.089 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:34 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:35.089 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:34 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/146082484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2026-02-13T10:39:35.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 -- 10.20.193.121:0/268722365 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc101590 msgr2=0x7f8dfc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/268722365 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc101590 0x7f8dfc101990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8df00099b0 tx=0x7f8df002f220 comp rx=0 tx=0).stop 2026-02-13T10:39:35.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 -- 10.20.193.121:0/268722365 shutdown_connections 2026-02-13T10:39:35.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/268722365 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8dfc102790 0x7f8dfc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/268722365 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc101590 0x7f8dfc101990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.178 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 -- 10.20.193.121:0/268722365 >> 10.20.193.121:0/268722365 conn(0x7f8dfc0fcd20 msgr2=0x7f8dfc0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 -- 10.20.193.121:0/268722365 shutdown_connections 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 -- 10.20.193.121:0/268722365 wait complete. 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 Processor -- start 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.173+0000 7f8e00bf6640 1 -- start start 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8e00bf6640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8dfc101590 0x7f8dfc19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8e00bf6640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc102790 0x7f8dfc19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8e00bf6640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10a280 con 0x7f8dfc102790 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8e00bf6640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10a100 con 0x7f8dfc101590 2026-02-13T10:39:35.179 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8df9d74640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc102790 0x7f8dfc19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8df9d74640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc102790 0x7f8dfc19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57016/0 (socket says 10.20.193.121:57016) 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8df9d74640 1 -- 10.20.193.121:0/2656457593 learned_addr learned my addr 10.20.193.121:0/2656457593 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8dfa575640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8dfc101590 0x7f8dfc19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8df9d74640 1 -- 10.20.193.121:0/2656457593 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8dfc101590 msgr2=0x7f8dfc19c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8df9d74640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8dfc101590 0x7f8dfc19c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8df9d74640 1 -- 10.20.193.121:0/2656457593 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc19d030 con 0x7f8dfc102790 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8dfa575640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8dfc101590 0x7f8dfc19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8df9d74640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc102790 0x7f8dfc19c9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8de400d900 tx=0x7f8de400ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8de37fe640 1 -- 10.20.193.121:0/2656457593 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f8de4014070 con 0x7f8dfc102790 2026-02-13T10:39:35.180 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8de37fe640 1 -- 10.20.193.121:0/2656457593 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f8de40044a0 con 0x7f8dfc102790 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc1a1a70 con 0x7f8dfc102790 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8de37fe640 1 -- 10.20.193.121:0/2656457593 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f8de4005020 con 0x7f8dfc102790 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc1a2080 con 0x7f8dfc102790 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.174+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dfc101990 con 0x7f8dfc102790 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.175+0000 7f8de37fe640 1 -- 10.20.193.121:0/2656457593 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f8de4005260 con 0x7f8dfc102790 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.175+0000 7f8de37fe640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8dd4077550 0x7f8dd4079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.175+0000 7f8de37fe640 1 -- 10.20.193.121:0/2656457593 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5459+0+0 (secure 0 0 0) 0x7f8de4099ae0 con 0x7f8dfc102790 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.175+0000 7f8dfa575640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8dd4077550 0x7f8dd4079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:35.181 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.176+0000 7f8dfa575640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8dd4077550 0x7f8dd4079a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097c0 tx=0x7f8df0005e50 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:35.182 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.177+0000 7f8de37fe640 1 -- 10.20.193.121:0/2656457593 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8de4066bd0 con 0x7f8dfc102790 2026-02-13T10:39:35.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:34 trial163 ceph-mon[24184]: pgmap v84: 65 pgs: 3 creating+peering, 24 unknown, 38 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 2.5 KiB/s wr, 3 op/s 2026-02-13T10:39:35.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:34 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2026-02-13T10:39:35.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:34 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1193641152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2026-02-13T10:39:35.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:34 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:35.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:34 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:35.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:34 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/146082484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2026-02-13T10:39:35.295 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.293+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) -- 0x7f8dfc1a2480 con 0x7f8dfc102790 2026-02-13T10:39:35.295 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.293+0000 7f8de37fe640 1 -- 10.20.193.121:0/2656457593 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v7) ==== 78+0+98 (secure 0 0 0) 0x7f8de406ba80 con 0x7f8dfc102790 2026-02-13T10:39:35.296 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8dd4077550 msgr2=0x7f8dd4079a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8dd4077550 0x7f8dd4079a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8df00097c0 tx=0x7f8df0005e50 comp rx=0 tx=0).stop 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc102790 msgr2=0x7f8dfc19c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc102790 0x7f8dfc19c9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8de400d900 tx=0x7f8de400ddd0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 shutdown_connections 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f8dd4077550 0x7f8dd4079a10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f8dfc102790 0x7f8dfc19c9b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 --2- 10.20.193.121:0/2656457593 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f8dfc101590 0x7f8dfc19c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 >> 10.20.193.121:0/2656457593 conn(0x7f8dfc0fcd20 msgr2=0x7f8dfc06aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 shutdown_connections 2026-02-13T10:39:35.297 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.294+0000 7f8e00bf6640 1 -- 10.20.193.121:0/2656457593 wait complete. 2026-02-13T10:39:35.302 INFO:teuthology.orchestra.run.trial121.stdout:true 2026-02-13T10:39:35.336 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2026-02-13T10:39:35.358 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T10:39:35.647 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:35.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- 10.20.193.121:0/3898410340 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 msgr2=0x7fd1b4113040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/3898410340 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b4113040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80099e0 tx=0x7fd1a802f260 comp rx=0 tx=0).stop 2026-02-13T10:39:35.759 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- 10.20.193.121:0/3898410340 shutdown_connections 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/3898410340 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b4113040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/3898410340 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd1b4077620 0x7fd1b4077a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- 10.20.193.121:0/3898410340 >> 10.20.193.121:0/3898410340 conn(0x7fd1b41003e0 msgr2=0x7fd1b4102800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- 10.20.193.121:0/3898410340 shutdown_connections 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- 10.20.193.121:0/3898410340 wait complete. 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 Processor -- start 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- start start 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd1b4077620 0x7fd1b41a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4115fb0 con 0x7fd1b4077f40 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.755+0000 7fd1bacc2640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4115e30 con 0x7fd1b4077620 2026-02-13T10:39:35.760 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1b94bf640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1b94bf640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b41a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57034/0 (socket says 10.20.193.121:57034) 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1b94bf640 1 -- 10.20.193.121:0/4207578572 learned_addr learned my addr 10.20.193.121:0/4207578572 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1b94bf640 1 -- 10.20.193.121:0/4207578572 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd1b4077620 msgr2=0x7fd1b41a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1b94bf640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd1b4077620 0x7fd1b41a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1b94bf640 1 -- 10.20.193.121:0/4207578572 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a56b0 con 0x7fd1b4077f40 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1b94bf640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b41a1320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8002410 tx=0x7fd1a8002e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1a2ffd640 1 -- 10.20.193.121:0/4207578572 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fd1a8038530 con 0x7fd1b4077f40 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1a2ffd640 1 -- 10.20.193.121:0/4207578572 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7fd1a8004420 con 0x7fd1b4077f40 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a59a0 con 0x7fd1b4077f40 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1a2ffd640 1 -- 10.20.193.121:0/4207578572 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7fd1a80419f0 con 0x7fd1b4077f40 2026-02-13T10:39:35.761 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a5f30 con 0x7fd1b4077f40 2026-02-13T10:39:35.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.756+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b4078d60 con 0x7fd1b4077f40 2026-02-13T10:39:35.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.757+0000 7fd1a2ffd640 1 -- 10.20.193.121:0/4207578572 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7fd1a8041ca0 con 0x7fd1b4077f40 2026-02-13T10:39:35.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.757+0000 7fd1a2ffd640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd1900775a0 0x7fd190079a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:35.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.757+0000 7fd1b9cc0640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd1900775a0 0x7fd190079a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:35.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.757+0000 7fd1a2ffd640 1 -- 10.20.193.121:0/4207578572 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5545+0+0 (secure 0 0 0) 0x7fd1a803e070 con 0x7fd1b4077f40 2026-02-13T10:39:35.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.758+0000 7fd1b9cc0640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd1900775a0 0x7fd190079a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400a9f0 tx=0x7fd1a4005cf0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:35.762 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.759+0000 7fd1a2ffd640 1 -- 10.20.193.121:0/4207578572 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd1a808b950 con 0x7fd1b4077f40 2026-02-13T10:39:35.861 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.859+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1b41a2140 con 0x7fd1b4077f40 2026-02-13T10:39:35.861 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.859+0000 7fd1a2ffd640 1 -- 10.20.193.121:0/4207578572 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4463 (secure 0 0 0) 0x7fd1a8090800 con 0x7fd1b4077f40 2026-02-13T10:39:35.862 INFO:teuthology.orchestra.run.trial121.stdout: 2026-02-13T10:39:35.862 INFO:teuthology.orchestra.run.trial121.stdout:{"epoch":8,"btime":"2026-02-13T10:39:35:322070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14612,"name":"cephfs.trial121.blaxtt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.121:6829/2967641829","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6828","nonce":2967641829},{"type":"v1","addr":"10.20.193.121:6829","nonce":2967641829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24305,"name":"cephfs.trial163.bxtdjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.163:6827/2686239026","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.163:6826","nonce":2686239026},{"type":"v1","addr":"10.20.193.163:6827","nonce":2686239026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2026-02-13T10:39:29.899884+0000","modified":"2026-02-13T10:39:35.322067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":14602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14602":{"gid":14602,"name":"cephfs.trial121.sfinrg","rank":0,"incarnation":8,"state":"up:replay","state_seq":2,"addr":"10.20.193.121:6827/1402594783","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.121:6826","nonce":1402594783},{"type":"v1","addr":"10.20.193.121:6827","nonce":1402594783}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2026-02-13T10:39:35.863 INFO:teuthology.orchestra.run.trial121.stderr:dumped fsmap epoch 8 2026-02-13T10:39:35.863 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd1900775a0 msgr2=0x7fd190079a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.863 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd1900775a0 0x7fd190079a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400a9f0 tx=0x7fd1a4005cf0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 msgr2=0x7fd1b41a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b41a1320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8002410 tx=0x7fd1a8002e00 comp rx=0 tx=0).stop 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 shutdown_connections 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7fd1900775a0 0x7fd190079a60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7fd1b4077f40 0x7fd1b41a1320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 --2- 10.20.193.121:0/4207578572 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7fd1b4077620 0x7fd1b41a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 >> 10.20.193.121:0/4207578572 conn(0x7fd1b41003e0 msgr2=0x7fd1b41007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 shutdown_connections 2026-02-13T10:39:35.864 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:35.860+0000 7fd1bacc2640 1 -- 10.20.193.121:0/4207578572 wait complete. 2026-02-13T10:39:35.899 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 8, 'max_mds': 1, 'flags': 18} 2026-02-13T10:39:35.900 INFO:teuthology.run_tasks:Running task ceph-fuse... 2026-02-13T10:39:35.959 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2026-02-13T10:39:35.959 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2026-02-13T10:39:35.960 INFO:tasks.ceph_fuse:client.0 config is {} 2026-02-13T10:39:35.960 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2026-02-13T10:39:35.960 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2026-02-13T10:39:35.960 INFO:tasks.ceph_fuse:client.1 config is {} 2026-02-13T10:39:35.960 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2026-02-13T10:39:35.960 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2026-02-13T10:39:35.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:35.960 DEBUG:teuthology.orchestra.run.trial121:> ip netns list 2026-02-13T10:39:35.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:35.978 DEBUG:teuthology.orchestra.run.trial121:> sudo ip link delete ceph-brx 2026-02-13T10:39:36.046 INFO:teuthology.orchestra.run.trial121.stderr:Cannot find device "ceph-brx" 2026-02-13T10:39:36.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:39:36.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:36.048 DEBUG:teuthology.orchestra.run.trial163:> ip netns list 2026-02-13T10:39:36.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:36.068 DEBUG:teuthology.orchestra.run.trial163:> sudo ip link delete ceph-brx 2026-02-13T10:39:36.137 INFO:teuthology.orchestra.run.trial163.stderr:Cannot find device "ceph-brx" 2026-02-13T10:39:36.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:39:36.139 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2026-02-13T10:39:36.139 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2026-02-13T10:39:36.139 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph fs ls 2026-02-13T10:39:36.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:36.208 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/2656457593' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: Dropping low affinity active daemon mds.cephfs.trial163.mabprx in favor of higher affinity standby. 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: Replacing daemon mds.cephfs.trial163.mabprx as rank 0 with standby daemon mds.cephfs.trial121.sfinrg 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: osdmap e43: 6 total, 6 up, 6 in 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:standby 2026-02-13T10:39:36.209 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: fsmap cephfs:1/1 {0=cephfs.trial121.sfinrg=up:replay} 2 up:standby 2026-02-13T10:39:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: pgmap v86: 65 pgs: 3 creating+peering, 62 active+clean; 579 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 2.3 KiB/s wr, 7 op/s 2026-02-13T10:39:36.210 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:35 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/4207578572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2026-02-13T10:39:36.290 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/2656457593' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: Dropping low affinity active daemon mds.cephfs.trial163.mabprx in favor of higher affinity standby. 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: Replacing daemon mds.cephfs.trial163.mabprx as rank 0 with standby daemon mds.cephfs.trial121.sfinrg 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2026-02-13T10:39:36.291 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: osdmap e43: 6 total, 6 up, 6 in 2026-02-13T10:39:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:standby 2026-02-13T10:39:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: fsmap cephfs:1/1 {0=cephfs.trial121.sfinrg=up:replay} 2 up:standby 2026-02-13T10:39:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: pgmap v86: 65 pgs: 3 creating+peering, 62 active+clean; 579 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 2.3 KiB/s wr, 7 op/s 2026-02-13T10:39:36.292 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:35 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/4207578572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2026-02-13T10:39:36.427 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:36.535 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 -- 10.20.193.121:0/2556555701 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 msgr2=0x7f7fb4102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:36.535 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 --2- 10.20.193.121:0/2556555701 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb4102ac0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7fa80099b0 tx=0x7f7fa802f220 comp rx=0 tx=0).stop 2026-02-13T10:39:36.535 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 -- 10.20.193.121:0/2556555701 shutdown_connections 2026-02-13T10:39:36.535 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 --2- 10.20.193.121:0/2556555701 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb4102ac0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.535 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 --2- 10.20.193.121:0/2556555701 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7fb4108660 0x7f7fb4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.535 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 -- 10.20.193.121:0/2556555701 >> 10.20.193.121:0/2556555701 conn(0x7f7fb40fe380 msgr2=0x7f7fb41007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:36.535 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 -- 10.20.193.121:0/2556555701 shutdown_connections 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.530+0000 7f7fbc372640 1 -- 10.20.193.121:0/2556555701 wait complete. 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 Processor -- start 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 -- start start 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb41a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 --2- >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7fb4108660 0x7f7fb41a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 -- --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb410f9c0 con 0x7f7fb4102660 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 -- --> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb410f840 con 0x7f7fb4108660 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fba0e7640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb41a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fba0e7640 1 --2- >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb41a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.121:3300/0 says I am v2:10.20.193.121:57048/0 (socket says 10.20.193.121:57048) 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fba0e7640 1 -- 10.20.193.121:0/1925945080 learned_addr learned my addr 10.20.193.121:0/1925945080 (peer_addr_for_me v2:10.20.193.121:0/0) 2026-02-13T10:39:36.536 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fb98e6640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7fb4108660 0x7f7fb41a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fba0e7640 1 -- 10.20.193.121:0/1925945080 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7fb4108660 msgr2=0x7f7fb41a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fba0e7640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7fb4108660 0x7f7fb41a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fba0e7640 1 -- 10.20.193.121:0/1925945080 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb419c6e0 con 0x7f7fb4102660 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fb98e6640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7fb4108660 0x7f7fb41a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fba0e7640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb41a2610 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d8d0 tx=0x7f7fa400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7f9b7fe640 1 -- 10.20.193.121:0/1925945080 <== mon.0 v2:10.20.193.121:3300/0 1 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7fa4014070 con 0x7f7fb4102660 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb419c9d0 con 0x7f7fb4102660 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7f9b7fe640 1 -- 10.20.193.121:0/1925945080 <== mon.0 v2:10.20.193.121:3300/0 2 ==== config(26 keys) ==== 1065+0+0 (secure 0 0 0) 0x7f7fa40044a0 con 0x7f7fb4102660 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7f9b7fe640 1 -- 10.20.193.121:0/1925945080 <== mon.0 v2:10.20.193.121:3300/0 3 ==== mon_map magic: 0 ==== 351+0+0 (secure 0 0 0) 0x7f7fa4002e60 con 0x7f7fb4102660 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.531+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb419cf30 con 0x7f7fb4102660 2026-02-13T10:39:36.537 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.532+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb4103da0 con 0x7f7fb4102660 2026-02-13T10:39:36.538 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.532+0000 7f7f9b7fe640 1 -- 10.20.193.121:0/1925945080 <== mon.0 v2:10.20.193.121:3300/0 4 ==== mgrmap(e 19) ==== 100119+0+0 (secure 0 0 0) 0x7f7fa400b840 con 0x7f7fb4102660 2026-02-13T10:39:36.538 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.533+0000 7f7f9b7fe640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7f90077540 0x7f7f90079a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T10:39:36.538 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.533+0000 7f7fb98e6640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7f90077540 0x7f7f90079a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T10:39:36.538 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.533+0000 7f7f9b7fe640 1 -- 10.20.193.121:0/1925945080 <== mon.0 v2:10.20.193.121:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5545+0+0 (secure 0 0 0) 0x7f7fa4099b50 con 0x7f7fb4102660 2026-02-13T10:39:36.538 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.533+0000 7f7fb98e6640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7f90077540 0x7f7f90079a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fa80097c0 tx=0x7f7fa80384d0 comp rx=0 tx=0).ready entity=mgr.14203 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T10:39:36.538 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.534+0000 7f7f9b7fe640 1 -- 10.20.193.121:0/1925945080 <== mon.0 v2:10.20.193.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7fa4099e20 con 0x7f7fb4102660 2026-02-13T10:39:36.636 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.634+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 --> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7fb4102ac0 con 0x7f7fb4102660 2026-02-13T10:39:36.636 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.634+0000 7f7f9b7fe640 1 -- 10.20.193.121:0/1925945080 <== mon.0 v2:10.20.193.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+83 (secure 0 0 0) 0x7f7fa4066bb0 con 0x7f7fb4102660 2026-02-13T10:39:36.637 INFO:teuthology.orchestra.run.trial121.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7f90077540 msgr2=0x7f7f90079a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7f90077540 0x7f7f90079a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fa80097c0 tx=0x7f7fa80384d0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 msgr2=0x7f7fb41a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb41a2610 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d8d0 tx=0x7f7fa400dda0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 shutdown_connections 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:6800/2058604417,v1:10.20.193.121:6801/2058604417] conn(0x7f7f90077540 0x7f7f90079a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.163:3300/0,v1:10.20.193.163:6789/0] conn(0x7f7fb4108660 0x7f7fb41a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 --2- 10.20.193.121:0/1925945080 >> [v2:10.20.193.121:3300/0,v1:10.20.193.121:6789/0] conn(0x7f7fb4102660 0x7f7fb41a2610 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 >> 10.20.193.121:0/1925945080 conn(0x7f7fb40fe380 msgr2=0x7f7fb40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T10:39:36.638 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 shutdown_connections 2026-02-13T10:39:36.639 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36.635+0000 7f7fbc372640 1 -- 10.20.193.121:0/1925945080 wait complete. 2026-02-13T10:39:36.672 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:self.client_remote.hostname = trial121.front.sepia.ceph.com 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:self.client.name = client.0 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:self.cephfs_name = None 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2026-02-13T10:39:36.673 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2026-02-13T10:39:36.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:36.674 DEBUG:teuthology.orchestra.run.trial121:> ip addr 2026-02-13T10:39:36.692 INFO:teuthology.orchestra.run.trial121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2026-02-13T10:39:36.693 INFO:teuthology.orchestra.run.trial121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2026-02-13T10:39:36.693 INFO:teuthology.orchestra.run.trial121.stdout: inet 127.0.0.1/8 scope host lo 2026-02-13T10:39:36.693 INFO:teuthology.orchestra.run.trial121.stdout: valid_lft forever preferred_lft forever 2026-02-13T10:39:36.693 INFO:teuthology.orchestra.run.trial121.stdout: inet6 ::1/128 scope host 2026-02-13T10:39:36.693 INFO:teuthology.orchestra.run.trial121.stdout: valid_lft forever preferred_lft forever 2026-02-13T10:39:36.693 INFO:teuthology.orchestra.run.trial121.stdout:2: enp7s0f4u2u2c2: mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000 2026-02-13T10:39:36.693 INFO:teuthology.orchestra.run.trial121.stdout: link/ether aa:3f:8e:5e:f9:b5 brd ff:ff:ff:ff:ff:ff 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: inet 169.254.3.1/24 brd 169.254.3.255 scope link dynamic noprefixroute enp7s0f4u2u2c2 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: valid_lft 863659sec preferred_lft 863659sec 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: inet6 fe80::ade3:66c0:f8e2:5ec4/64 scope link noprefixroute 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: valid_lft forever preferred_lft forever 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout:3: enp1s0f0: mtu 1500 qdisc mq state UP group default qlen 1000 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: link/ether 90:5a:08:77:61:fc brd ff:ff:ff:ff:ff:ff 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: inet 10.20.193.121/20 brd 10.20.207.255 scope global dynamic noprefixroute enp1s0f0 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: valid_lft 559sec preferred_lft 559sec 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: inet6 fe80::925a:8ff:fe77:61fc/64 scope link 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout: valid_lft forever preferred_lft forever 2026-02-13T10:39:36.694 INFO:teuthology.orchestra.run.trial121.stdout:4: enp1s0f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2026-02-13T10:39:36.695 INFO:teuthology.orchestra.run.trial121.stdout: link/ether 90:5a:08:77:61:fd brd ff:ff:ff:ff:ff:ff 2026-02-13T10:39:36.695 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.159.254/20 2026-02-13T10:39:36.696 DEBUG:teuthology.orchestra.run.trial121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-13T10:39:36.696 DEBUG:teuthology.orchestra.run.trial121:> set -e 2026-02-13T10:39:36.696 DEBUG:teuthology.orchestra.run.trial121:> sudo ip link add name ceph-brx type bridge 2026-02-13T10:39:36.696 DEBUG:teuthology.orchestra.run.trial121:> sudo ip addr flush dev ceph-brx 2026-02-13T10:39:36.696 DEBUG:teuthology.orchestra.run.trial121:> sudo ip link set ceph-brx up 2026-02-13T10:39:36.696 DEBUG:teuthology.orchestra.run.trial121:> sudo ip addr add 192.168.159.254/20 brd 192.168.159.255 dev ceph-brx 2026-02-13T10:39:36.697 DEBUG:teuthology.orchestra.run.trial121:> ') 2026-02-13T10:39:36.768 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-13T10:39:36.839 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-13T10:39:36.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:36.844 DEBUG:teuthology.orchestra.run.trial121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2026-02-13T10:39:36.909 INFO:teuthology.orchestra.run.trial121.stdout:1 2026-02-13T10:39:36.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:36.911 DEBUG:teuthology.orchestra.run.trial121:> ip r 2026-02-13T10:39:36.967 INFO:teuthology.orchestra.run.trial121.stdout:default via 10.20.192.1 dev enp1s0f0 proto dhcp src 10.20.193.121 metric 102 2026-02-13T10:39:36.967 INFO:teuthology.orchestra.run.trial121.stdout:10.20.192.0/20 dev enp1s0f0 proto kernel scope link src 10.20.193.121 metric 102 2026-02-13T10:39:36.967 INFO:teuthology.orchestra.run.trial121.stdout:169.254.3.0/24 dev enp7s0f4u2u2c2 proto kernel scope link src 169.254.3.1 metric 101 2026-02-13T10:39:36.967 INFO:teuthology.orchestra.run.trial121.stdout:192.168.144.0/20 dev ceph-brx proto kernel scope link src 192.168.159.254 2026-02-13T10:39:36.968 DEBUG:teuthology.orchestra.run.trial121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-13T10:39:36.968 DEBUG:teuthology.orchestra.run.trial121:> set -e 2026-02-13T10:39:36.968 DEBUG:teuthology.orchestra.run.trial121:> sudo iptables -A FORWARD -o enp1s0f0 -i ceph-brx -j ACCEPT 2026-02-13T10:39:36.968 DEBUG:teuthology.orchestra.run.trial121:> sudo iptables -A FORWARD -i enp1s0f0 -o ceph-brx -j ACCEPT 2026-02-13T10:39:36.968 DEBUG:teuthology.orchestra.run.trial121:> sudo iptables -t nat -A POSTROUTING -s 192.168.159.254/20 -o enp1s0f0 -j MASQUERADE 2026-02-13T10:39:36.968 DEBUG:teuthology.orchestra.run.trial121:> ') 2026-02-13T10:39:37.041 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-13T10:39:37.171 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-13T10:39:37.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:37.176 DEBUG:teuthology.orchestra.run.trial121:> ip netns list 2026-02-13T10:39:37.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:37.235 DEBUG:teuthology.orchestra.run.trial121:> ip netns list-id 2026-02-13T10:39:37.290 DEBUG:teuthology.orchestra.run.trial121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-13T10:39:37.290 DEBUG:teuthology.orchestra.run.trial121:> set -e 2026-02-13T10:39:37.290 DEBUG:teuthology.orchestra.run.trial121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-13T10:39:37.290 DEBUG:teuthology.orchestra.run.trial121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2026-02-13T10:39:37.290 DEBUG:teuthology.orchestra.run.trial121:> ') 2026-02-13T10:39:37.365 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-13T10:39:37.386 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-13T10:39:37.390 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.144.1/20 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> set -e 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.144.1/20 brd 192.168.159.255 dev veth0 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.159.254 2026-02-13T10:39:37.391 DEBUG:teuthology.orchestra.run.trial121:> ') 2026-02-13T10:39:37.465 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-13T10:39:37.521 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-13T10:39:37.526 DEBUG:teuthology.orchestra.run.trial121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-13T10:39:37.526 DEBUG:teuthology.orchestra.run.trial121:> set -e 2026-02-13T10:39:37.526 DEBUG:teuthology.orchestra.run.trial121:> sudo ip link set brx.0 up 2026-02-13T10:39:37.526 DEBUG:teuthology.orchestra.run.trial121:> sudo ip link set dev brx.0 master ceph-brx 2026-02-13T10:39:37.527 DEBUG:teuthology.orchestra.run.trial121:> ') 2026-02-13T10:39:37.600 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-13T10:39:37.607 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:37 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:reconnect 2026-02-13T10:39:37.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:37 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.163:6824/1586652898,v1:10.20.193.163:6825/1586652898] up:boot 2026-02-13T10:39:37.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:37 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.121:6828/2967641829,v1:10.20.193.121:6829/2967641829] up:standby 2026-02-13T10:39:37.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:37 trial121 ceph-mon[19159]: fsmap cephfs:1/1 {0=cephfs.trial121.sfinrg=up:reconnect} 3 up:standby 2026-02-13T10:39:37.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:37 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial163.mabprx"}]: dispatch 2026-02-13T10:39:37.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:37 trial121 ceph-mon[19159]: from='client.? 10.20.193.121:0/1925945080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2026-02-13T10:39:37.608 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:37 trial121 ceph-mon[19159]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:37.629 INFO:teuthology.orchestra.run.trial121.stderr:2026-02-13T10:39:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-13T10:39:37.636 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2026-02-13T10:39:37.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2026-02-13T10:39:37.636 DEBUG:teuthology.orchestra.run.trial121:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2026-02-13T10:39:37.692 INFO:teuthology.orchestra.run.trial121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2026-02-13T10:39:37.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2026-02-13T10:39:37.693 DEBUG:teuthology.orchestra.run.trial121:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2026-02-13T10:39:37.749 DEBUG:teuthology.orchestra.run.trial121:> sudo modprobe fuse 2026-02-13T10:39:37.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:37 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:reconnect 2026-02-13T10:39:37.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:37 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.163:6824/1586652898,v1:10.20.193.163:6825/1586652898] up:boot 2026-02-13T10:39:37.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:37 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.121:6828/2967641829,v1:10.20.193.121:6829/2967641829] up:standby 2026-02-13T10:39:37.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:37 trial163 ceph-mon[24184]: fsmap cephfs:1/1 {0=cephfs.trial121.sfinrg=up:reconnect} 3 up:standby 2026-02-13T10:39:37.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:37 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' cmd=[{"prefix": "mds metadata", "who": "cephfs.trial163.mabprx"}]: dispatch 2026-02-13T10:39:37.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:37 trial163 ceph-mon[24184]: from='client.? 10.20.193.121:0/1925945080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2026-02-13T10:39:37.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:37 trial163 ceph-mon[24184]: from='mgr.14203 10.20.193.121:0/1258427745' entity='mgr.trial121.tkoihk' 2026-02-13T10:39:37.814 DEBUG:teuthology.orchestra.run.trial121:> cat /proc/self/mounts | awk '{print $2}' 2026-02-13T10:39:37.871 INFO:teuthology.orchestra.run.trial121.stdout:/proc 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/sys 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/dev 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/security 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/dev/shm 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/dev/pts 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/run 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/cgroup 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/pstore 2026-02-13T10:39:37.872 INFO:teuthology.orchestra.run.trial121.stdout:/sys/firmware/efi/efivars 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/bpf 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/config 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/ 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/selinux 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/proc/sys/fs/binfmt_misc 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/dev/hugepages 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/dev/mqueue 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/debug 2026-02-13T10:39:37.873 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/tracing 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/fuse/connections 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-sysctl.service 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/boot/efi 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/nfs/rpc_pipefs 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/run/user/1000 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/ceph 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay 2026-02-13T10:39:37.874 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/57c4d4d3bdcde99ec80a35766c3765b43b2889954a8242d3dd17e81c4408dd37/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/66654fc944c8d68eface5229f0d5d9ca24a3437a980f411b4c808d41bd9e2da8/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/run/user/0 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/proc/sys/fs/binfmt_misc 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/3eaf86d4abb833328dfe70f91cd127324ad92058e4378f6add3e0fa273f6df4d/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/2ebd14db9dcf58435c3d4d72541e37ef6754aaf0b30f7054c8ec2db4a92e275c/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/cfbbf19725d593f106628c1b92c0b3947b1d38b541754b2b59f162872ed16a18/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/b09083470862abc161475736ffaa5e93a261b44759f43b9e83975abb7e4b8872/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/d1ccb88d504978f074ceb52dbc5e4c6c1247635c1925d142e3c728a0c20a7026/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/668fb6cad529a81c6297eaccbb34377516635632ffbab3357427c4d555f0dfc7/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/a824a0b2fcae963160b7b759a0bcfc1296f0b90c882de76cb828680fdccbf3b1/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/2f4f3df47f688e1a404c2bef776a049f33d2608c225c196951a424538fcc954d/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/c0aa10c931e152a06915deb69b2390b0d22023cdfee96ad42ea02e073d7408d2/merged 2026-02-13T10:39:37.875 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/cfe2d737e81eeee968c3ce0d872bd7fff7b9fcc2f9bb6215f2267633c367ef76/merged 2026-02-13T10:39:37.876 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/6e5a8c3e8e0167cbc53a1b340c82ac01c80c3adde803b9e10e98f11c28331170/merged 2026-02-13T10:39:37.876 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns 2026-02-13T10:39:37.876 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-13T10:39:37.876 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-13T10:39:37.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:37.876 DEBUG:teuthology.orchestra.run.trial121:> ls /sys/fs/fuse/connections 2026-02-13T10:39:37.928 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2026-02-13T10:39:37.928 DEBUG:teuthology.orchestra.run.trial121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2026-02-13T10:39:37.971 DEBUG:teuthology.orchestra.run.trial121:> sudo modprobe fuse 2026-02-13T10:39:37.995 DEBUG:teuthology.orchestra.run.trial121:> cat /proc/self/mounts | awk '{print $2}' 2026-02-13T10:39:38.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial121.stderr:2026-02-13T10:39:38.048+0000 7efcea9d9580 -1 init, newargv = 0x55769e1ce6b0 newargc=13 2026-02-13T10:39:38.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial121.stderr:2026-02-13T10:39:38.048+0000 7efcea9d9580 -1 init, args.argv = 0x55769e1e1270 args.argc=4 2026-02-13T10:39:38.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial121.stderr:ceph-fuse[68952]: starting ceph client 2026-02-13T10:39:38.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial121.stderr:ceph-fuse[68952]: probably no MDS server is up? 2026-02-13T10:39:38.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial121.stderr:ceph-fuse[68952]: ceph mount failed with (65536) Unknown error 65536 2026-02-13T10:39:38.053 INFO:teuthology.orchestra.run.trial121.stdout:/proc 2026-02-13T10:39:38.053 INFO:teuthology.orchestra.run.trial121.stdout:/sys 2026-02-13T10:39:38.053 INFO:teuthology.orchestra.run.trial121.stdout:/dev 2026-02-13T10:39:38.053 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/security 2026-02-13T10:39:38.053 INFO:teuthology.orchestra.run.trial121.stdout:/dev/shm 2026-02-13T10:39:38.053 INFO:teuthology.orchestra.run.trial121.stdout:/dev/pts 2026-02-13T10:39:38.053 INFO:teuthology.orchestra.run.trial121.stdout:/run 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/cgroup 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/pstore 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/firmware/efi/efivars 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/bpf 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/config 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/ 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/selinux 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/proc/sys/fs/binfmt_misc 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/dev/hugepages 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/dev/mqueue 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/debug 2026-02-13T10:39:38.054 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/tracing 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/fuse/connections 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-sysctl.service 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/boot/efi 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/nfs/rpc_pipefs 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/run/user/1000 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/ceph 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/57c4d4d3bdcde99ec80a35766c3765b43b2889954a8242d3dd17e81c4408dd37/merged 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/66654fc944c8d68eface5229f0d5d9ca24a3437a980f411b4c808d41bd9e2da8/merged 2026-02-13T10:39:38.055 INFO:teuthology.orchestra.run.trial121.stdout:/run/user/0 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/proc/sys/fs/binfmt_misc 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/3eaf86d4abb833328dfe70f91cd127324ad92058e4378f6add3e0fa273f6df4d/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/2ebd14db9dcf58435c3d4d72541e37ef6754aaf0b30f7054c8ec2db4a92e275c/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/cfbbf19725d593f106628c1b92c0b3947b1d38b541754b2b59f162872ed16a18/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/b09083470862abc161475736ffaa5e93a261b44759f43b9e83975abb7e4b8872/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/d1ccb88d504978f074ceb52dbc5e4c6c1247635c1925d142e3c728a0c20a7026/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/668fb6cad529a81c6297eaccbb34377516635632ffbab3357427c4d555f0dfc7/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/a824a0b2fcae963160b7b759a0bcfc1296f0b90c882de76cb828680fdccbf3b1/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/2f4f3df47f688e1a404c2bef776a049f33d2608c225c196951a424538fcc954d/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/c0aa10c931e152a06915deb69b2390b0d22023cdfee96ad42ea02e073d7408d2/merged 2026-02-13T10:39:38.056 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/cfe2d737e81eeee968c3ce0d872bd7fff7b9fcc2f9bb6215f2267633c367ef76/merged 2026-02-13T10:39:38.057 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/6e5a8c3e8e0167cbc53a1b340c82ac01c80c3adde803b9e10e98f11c28331170/merged 2026-02-13T10:39:38.057 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns 2026-02-13T10:39:38.057 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-13T10:39:38.057 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-13T10:39:38.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:38.058 DEBUG:teuthology.orchestra.run.trial121:> ls /sys/fs/fuse/connections 2026-02-13T10:39:38.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial121.stderr:daemon-helper: command failed with exit status 1 2026-02-13T10:39:38.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:38 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:rejoin 2026-02-13T10:39:38.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:38 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.163:6826/2686239026,v1:10.20.193.163:6827/2686239026] up:standby 2026-02-13T10:39:38.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:38 trial121 ceph-mon[19159]: fsmap cephfs:1/1 {0=cephfs.trial121.sfinrg=up:rejoin} 3 up:standby 2026-02-13T10:39:38.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:38 trial121 ceph-mon[19159]: daemon mds.cephfs.trial121.sfinrg is now active in filesystem cephfs as rank 0 2026-02-13T10:39:38.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:38 trial121 ceph-mon[19159]: pgmap v87: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 461 B/s rd, 2.1 KiB/s wr, 7 op/s 2026-02-13T10:39:38.711 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:38 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:rejoin 2026-02-13T10:39:38.711 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:38 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.163:6826/2686239026,v1:10.20.193.163:6827/2686239026] up:standby 2026-02-13T10:39:38.711 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:38 trial163 ceph-mon[24184]: fsmap cephfs:1/1 {0=cephfs.trial121.sfinrg=up:rejoin} 3 up:standby 2026-02-13T10:39:38.711 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:38 trial163 ceph-mon[24184]: daemon mds.cephfs.trial121.sfinrg is now active in filesystem cephfs as rank 0 2026-02-13T10:39:38.711 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:38 trial163 ceph-mon[24184]: pgmap v87: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 461 B/s rd, 2.1 KiB/s wr, 7 op/s 2026-02-13T10:39:39.220 DEBUG:teuthology.orchestra.run.trial121:> sudo modprobe fuse 2026-02-13T10:39:39.248 DEBUG:teuthology.orchestra.run.trial121:> cat /proc/self/mounts | awk '{print $2}' 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/proc 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/sys 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/dev 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/security 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/dev/shm 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/dev/pts 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/run 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/cgroup 2026-02-13T10:39:39.306 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/pstore 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/sys/firmware/efi/efivars 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/bpf 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/config 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/ 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/selinux 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/proc/sys/fs/binfmt_misc 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/dev/hugepages 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/dev/mqueue 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/debug 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/sys/kernel/tracing 2026-02-13T10:39:39.307 INFO:teuthology.orchestra.run.trial121.stdout:/sys/fs/fuse/connections 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-sysctl.service 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/boot/efi 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/nfs/rpc_pipefs 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/run/user/1000 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/ceph 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/57c4d4d3bdcde99ec80a35766c3765b43b2889954a8242d3dd17e81c4408dd37/merged 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/66654fc944c8d68eface5229f0d5d9ca24a3437a980f411b4c808d41bd9e2da8/merged 2026-02-13T10:39:39.308 INFO:teuthology.orchestra.run.trial121.stdout:/run/user/0 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/proc/sys/fs/binfmt_misc 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/3eaf86d4abb833328dfe70f91cd127324ad92058e4378f6add3e0fa273f6df4d/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/2ebd14db9dcf58435c3d4d72541e37ef6754aaf0b30f7054c8ec2db4a92e275c/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/cfbbf19725d593f106628c1b92c0b3947b1d38b541754b2b59f162872ed16a18/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/b09083470862abc161475736ffaa5e93a261b44759f43b9e83975abb7e4b8872/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/d1ccb88d504978f074ceb52dbc5e4c6c1247635c1925d142e3c728a0c20a7026/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/668fb6cad529a81c6297eaccbb34377516635632ffbab3357427c4d555f0dfc7/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/a824a0b2fcae963160b7b759a0bcfc1296f0b90c882de76cb828680fdccbf3b1/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/2f4f3df47f688e1a404c2bef776a049f33d2608c225c196951a424538fcc954d/merged 2026-02-13T10:39:39.309 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/c0aa10c931e152a06915deb69b2390b0d22023cdfee96ad42ea02e073d7408d2/merged 2026-02-13T10:39:39.310 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/cfe2d737e81eeee968c3ce0d872bd7fff7b9fcc2f9bb6215f2267633c367ef76/merged 2026-02-13T10:39:39.310 INFO:teuthology.orchestra.run.trial121.stdout:/var/lib/containers/storage/overlay/6e5a8c3e8e0167cbc53a1b340c82ac01c80c3adde803b9e10e98f11c28331170/merged 2026-02-13T10:39:39.310 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns 2026-02-13T10:39:39.310 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-13T10:39:39.310 INFO:teuthology.orchestra.run.trial121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-13T10:39:39.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-13T10:39:39.311 DEBUG:teuthology.orchestra.run.trial121:> ls /sys/fs/fuse/connections 2026-02-13T10:39:39.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:39:39.362 INFO:tasks.cephfs.fuse_mount:mount command failed. 2026-02-13T10:39:39.362 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa/tasks/ceph_fuse.py", line 234, in task mount_x.mount(mntopts=config.get('mntopts', []), mntargs=mntargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa/tasks/cephfs/fuse_mount.py", line 52, in mount return self._mount(mntopts, mntargs, check_status) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa/tasks/cephfs/fuse_mount.py", line 68, in _mount retval = self._run_mount_cmd(mntopts, mntargs, check_status) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa/tasks/cephfs/fuse_mount.py", line 94, in _run_mount_cmd return self._wait_and_record_our_fuse_conn( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_79a8b77b79a4b70f9c11cf81d24d449d40916c58/qa/tasks/cephfs/fuse_mount.py", line 178, in _wait_and_record_our_fuse_conn self.fuse_daemon.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial121 with status 1: "(cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0)" 2026-02-13T10:39:39.366 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2026-02-13T10:39:39.383 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2026-02-13T10:39:39.400 INFO:tasks.cephadm:Teardown begin 2026-02-13T10:39:39.401 DEBUG:teuthology.orchestra.run.trial121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T10:39:39.427 DEBUG:teuthology.orchestra.run.trial163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T10:39:39.454 INFO:tasks.cephadm:Disabling cephadm mgr module 2026-02-13T10:39:39.454 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 -- ceph mgr module disable cephadm 2026-02-13T10:39:39.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:39 trial121 ceph-mon[19159]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2026-02-13T10:39:39.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:39 trial121 ceph-mon[19159]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:active 2026-02-13T10:39:39.708 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:39 trial121 ceph-mon[19159]: fsmap cephfs:1 {0=cephfs.trial121.sfinrg=up:active} 3 up:standby 2026-02-13T10:39:39.756 INFO:teuthology.orchestra.run.trial121.stderr:Inferring config /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/mon.trial121/config 2026-02-13T10:39:39.774 INFO:teuthology.orchestra.run.trial121.stderr:Error: statfs /etc/ceph/ceph.client.admin.keyring: no such file or directory 2026-02-13T10:39:39.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:39 trial163 ceph-mon[24184]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2026-02-13T10:39:39.790 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:39 trial163 ceph-mon[24184]: mds.? [v2:10.20.193.121:6826/1402594783,v1:10.20.193.121:6827/1402594783] up:active 2026-02-13T10:39:39.791 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:39 trial163 ceph-mon[24184]: fsmap cephfs:1 {0=cephfs.trial121.sfinrg=up:active} 3 up:standby 2026-02-13T10:39:39.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2026-02-13T10:39:39.792 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2026-02-13T10:39:39.792 DEBUG:teuthology.orchestra.run.trial121:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-13T10:39:39.809 DEBUG:teuthology.orchestra.run.trial163:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-13T10:39:39.827 INFO:tasks.cephadm:Stopping all daemons... 2026-02-13T10:39:39.828 INFO:tasks.cephadm.mon.trial121:Stopping mon.trial121... 2026-02-13T10:39:39.828 DEBUG:teuthology.orchestra.run.trial121:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121 2026-02-13T10:39:40.043 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:39 trial121 systemd[1]: Stopping Ceph mon.trial121 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:39:40.043 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:39 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121[19155]: 2026-02-13T10:39:39.951+0000 7faa4df25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.trial121 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:39:40.044 INFO:journalctl@ceph.mon.trial121.trial121.stdout:Feb 13 10:39:39 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial121[19155]: 2026-02-13T10:39:39.951+0000 7faa4df25640 -1 mon.trial121@0(leader) e2 *** Got Signal Terminated *** 2026-02-13T10:39:40.134 DEBUG:teuthology.orchestra.run.trial121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial121.service' 2026-02-13T10:39:40.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:39:40.185 INFO:tasks.cephadm.mon.trial121:Stopped mon.trial121 2026-02-13T10:39:40.185 INFO:tasks.cephadm.mon.trial163:Stopping mon.trial163... 2026-02-13T10:39:40.185 DEBUG:teuthology.orchestra.run.trial163:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial163 2026-02-13T10:39:40.528 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:40 trial163 systemd[1]: Stopping Ceph mon.trial163 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:39:40.528 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:40 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial163[24180]: 2026-02-13T10:39:40.283+0000 7f2a40f29640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.trial163 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:39:40.528 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:40 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial163[24180]: 2026-02-13T10:39:40.283+0000 7f2a40f29640 -1 mon.trial163@1(peon) e2 *** Got Signal Terminated *** 2026-02-13T10:39:40.528 INFO:journalctl@ceph.mon.trial163.trial163.stdout:Feb 13 10:39:40 trial163 podman[47833]: 2026-02-13 10:39:40.52660993 +0000 UTC m=+0.255937334 container died 645ee540e429e26221aba7e4c4e4becdfb315109b0f859519bc817f3d7674909 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-mon-trial163, ceph=True, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, org.label-schema.build-date=20260202, CEPH_REF=squid, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T10:39:40.621 DEBUG:teuthology.orchestra.run.trial163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@mon.trial163.service' 2026-02-13T10:39:40.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:39:40.670 INFO:tasks.cephadm.mon.trial163:Stopped mon.trial163 2026-02-13T10:39:40.670 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2026-02-13T10:39:40.670 DEBUG:teuthology.orchestra.run.trial121:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.0 2026-02-13T10:39:40.958 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:40 trial121 systemd[1]: Stopping Ceph osd.0 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:39:40.958 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:40 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0[40833]: 2026-02-13T10:39:40.768+0000 7f70042b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:39:40.959 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:40 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0[40833]: 2026-02-13T10:39:40.768+0000 7f70042b9640 -1 osd.0 43 *** Got signal Terminated *** 2026-02-13T10:39:40.959 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:40 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0[40833]: 2026-02-13T10:39:40.768+0000 7f70042b9640 -1 osd.0 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69492]: 2026-02-13 10:39:46.294353193 +0000 UTC m=+5.538810760 container died 379a9e166bb98ce2980943414fec93bde6d349c7a518bcaedc69b5c6b7750ee5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260202, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, ceph=True, org.label-schema.license=GPLv2, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_REF=squid, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69492]: 2026-02-13 10:39:46.303547531 +0000 UTC m=+5.548005108 container remove 379a9e166bb98ce2980943414fec93bde6d349c7a518bcaedc69b5c6b7750ee5 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0, OSD_FLAVOR=default, org.label-schema.build-date=20260202, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, ceph=True, CEPH_REF=squid, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 bash[69492]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69626]: 2026-02-13 10:39:46.443615746 +0000 UTC m=+0.016590101 container create e8f3227c1f88db247fee685a4b09e280b05dc4fe32a6ad9225f437438c8191d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0-deactivate, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, ceph=True, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, CEPH_REF=squid, org.label-schema.build-date=20260202, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69626]: 2026-02-13 10:39:46.476615903 +0000 UTC m=+0.049590268 container init e8f3227c1f88db247fee685a4b09e280b05dc4fe32a6ad9225f437438c8191d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0-deactivate, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, CEPH_REF=squid, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260202, OSD_FLAVOR=default) 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69626]: 2026-02-13 10:39:46.47914387 +0000 UTC m=+0.052118245 container start e8f3227c1f88db247fee685a4b09e280b05dc4fe32a6ad9225f437438c8191d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20260202, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=squid, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, io.buildah.version=1.41.3) 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69626]: 2026-02-13 10:39:46.479858425 +0000 UTC m=+0.052832800 container attach e8f3227c1f88db247fee685a4b09e280b05dc4fe32a6ad9225f437438c8191d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0-deactivate, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_REF=squid, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, OSD_FLAVOR=default, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260202, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS) 2026-02-13T10:39:46.602 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69626]: 2026-02-13 10:39:46.436181672 +0000 UTC m=+0.009156027 image pull 4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4 2026-02-13T10:39:46.603 INFO:journalctl@ceph.osd.0.trial121.stdout:Feb 13 10:39:46 trial121 podman[69626]: 2026-02-13 10:39:46.600347999 +0000 UTC m=+0.173322364 container died e8f3227c1f88db247fee685a4b09e280b05dc4fe32a6ad9225f437438c8191d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-0-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, CEPH_REF=squid, org.label-schema.build-date=20260202, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-13T10:39:46.639 DEBUG:teuthology.orchestra.run.trial121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.0.service' 2026-02-13T10:39:46.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:39:46.687 INFO:tasks.cephadm.osd.0:Stopped osd.0 2026-02-13T10:39:46.687 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2026-02-13T10:39:46.687 DEBUG:teuthology.orchestra.run.trial121:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.1 2026-02-13T10:39:46.890 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:46 trial121 systemd[1]: Stopping Ceph osd.1 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:39:46.890 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:46 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1[47880]: 2026-02-13T10:39:46.789+0000 7f59f7789640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:39:46.890 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:46 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1[47880]: 2026-02-13T10:39:46.789+0000 7f59f7789640 -1 osd.1 43 *** Got signal Terminated *** 2026-02-13T10:39:46.891 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:46 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1[47880]: 2026-02-13T10:39:46.789+0000 7f59f7789640 -1 osd.1 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T10:39:52.459 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 podman[69766]: 2026-02-13 10:39:52.15432926 +0000 UTC m=+5.377308857 container died 2a29d84c07b3a84d9c9f90d8c2fd612aaa823bda36337e92b777b1d0b52895e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, OSD_FLAVOR=default, CEPH_REF=squid, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260202, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2026-02-13T10:39:52.459 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 podman[69766]: 2026-02-13 10:39:52.164065382 +0000 UTC m=+5.387044959 container remove 2a29d84c07b3a84d9c9f90d8c2fd612aaa823bda36337e92b777b1d0b52895e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, CEPH_REF=squid, OSD_FLAVOR=default, ceph=True, org.label-schema.build-date=20260202, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T10:39:52.459 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 bash[69766]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1 2026-02-13T10:39:52.459 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 podman[69898]: 2026-02-13 10:39:52.303785612 +0000 UTC m=+0.015531398 container create 95924c3b6fe92352d932502e1532ffc2b0c2d3a0833f32c2bc7d8ee090b54234 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1-deactivate, ceph=True, io.buildah.version=1.41.3, org.label-schema.build-date=20260202, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, CEPH_REF=squid, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2026-02-13T10:39:52.459 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 podman[69898]: 2026-02-13 10:39:52.340014185 +0000 UTC m=+0.051759991 container init 95924c3b6fe92352d932502e1532ffc2b0c2d3a0833f32c2bc7d8ee090b54234 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1-deactivate, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, CEPH_REF=squid, org.label-schema.build-date=20260202) 2026-02-13T10:39:52.459 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 podman[69898]: 2026-02-13 10:39:52.342598296 +0000 UTC m=+0.054344093 container start 95924c3b6fe92352d932502e1532ffc2b0c2d3a0833f32c2bc7d8ee090b54234 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1-deactivate, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=squid, org.label-schema.build-date=20260202, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-13T10:39:52.459 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 podman[69898]: 2026-02-13 10:39:52.343104986 +0000 UTC m=+0.054850782 container attach 95924c3b6fe92352d932502e1532ffc2b0c2d3a0833f32c2bc7d8ee090b54234 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-1-deactivate, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=squid, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, OSD_FLAVOR=default, io.buildah.version=1.41.3, org.label-schema.build-date=20260202, org.label-schema.schema-version=1.0, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869) 2026-02-13T10:39:52.460 INFO:journalctl@ceph.osd.1.trial121.stdout:Feb 13 10:39:52 trial121 podman[69898]: 2026-02-13 10:39:52.297774627 +0000 UTC m=+0.009520423 image pull 4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4 2026-02-13T10:39:52.494 DEBUG:teuthology.orchestra.run.trial121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.1.service' 2026-02-13T10:39:52.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:39:52.540 INFO:tasks.cephadm.osd.1:Stopped osd.1 2026-02-13T10:39:52.540 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2026-02-13T10:39:52.540 DEBUG:teuthology.orchestra.run.trial121:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.2 2026-02-13T10:39:52.958 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:52 trial121 systemd[1]: Stopping Ceph osd.2 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:39:52.958 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:52 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2[54994]: 2026-02-13T10:39:52.680+0000 7f5424df6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:39:52.958 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:52 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2[54994]: 2026-02-13T10:39:52.680+0000 7f5424df6640 -1 osd.2 43 *** Got signal Terminated *** 2026-02-13T10:39:52.958 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:52 trial121 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2[54994]: 2026-02-13T10:39:52.680+0000 7f5424df6640 -1 osd.2 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T10:39:58.315 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70031]: 2026-02-13 10:39:58.012645211 +0000 UTC m=+5.344552845 container died 82f925ca8d68eb14f731d1850e41bf74836c3d2f608f6a24602e14a223c96e74 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2, OSD_FLAVOR=default, io.buildah.version=1.41.3, org.label-schema.build-date=20260202, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=squid, org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, org.label-schema.license=GPLv2, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T10:39:58.315 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70031]: 2026-02-13 10:39:58.022807677 +0000 UTC m=+5.354715321 container remove 82f925ca8d68eb14f731d1850e41bf74836c3d2f608f6a24602e14a223c96e74 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2, org.label-schema.vendor=CentOS, CEPH_REF=squid, io.buildah.version=1.41.3, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2, org.label-schema.build-date=20260202, org.label-schema.schema-version=1.0, ceph=True, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T10:39:58.315 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 bash[70031]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2 2026-02-13T10:39:58.315 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70168]: 2026-02-13 10:39:58.162192494 +0000 UTC m=+0.016314499 container create 6e7c17b1dc7fe9ccd474ecd00cd23ebc40348c86e23eb74a199fb4b9a44a1a43 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.41.3, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_REF=squid, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.build-date=20260202, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, ceph=True, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-13T10:39:58.315 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70168]: 2026-02-13 10:39:58.189570573 +0000 UTC m=+0.043692578 container init 6e7c17b1dc7fe9ccd474ecd00cd23ebc40348c86e23eb74a199fb4b9a44a1a43 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2-deactivate, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260202, CEPH_REF=squid, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-13T10:39:58.315 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70168]: 2026-02-13 10:39:58.192285428 +0000 UTC m=+0.046407443 container start 6e7c17b1dc7fe9ccd474ecd00cd23ebc40348c86e23eb74a199fb4b9a44a1a43 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2-deactivate, org.label-schema.license=GPLv2, OSD_FLAVOR=default, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_REF=squid, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260202, org.label-schema.vendor=CentOS) 2026-02-13T10:39:58.315 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70168]: 2026-02-13 10:39:58.192875264 +0000 UTC m=+0.046997279 container attach 6e7c17b1dc7fe9ccd474ecd00cd23ebc40348c86e23eb74a199fb4b9a44a1a43 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2-deactivate, org.label-schema.vendor=CentOS, OSD_FLAVOR=default, ceph=True, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260202, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=squid, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-13T10:39:58.316 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70168]: 2026-02-13 10:39:58.156151388 +0000 UTC m=+0.010273393 image pull 4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4 2026-02-13T10:39:58.316 INFO:journalctl@ceph.osd.2.trial121.stdout:Feb 13 10:39:58 trial121 podman[70168]: 2026-02-13 10:39:58.313487064 +0000 UTC m=+0.167609069 container died 6e7c17b1dc7fe9ccd474ecd00cd23ebc40348c86e23eb74a199fb4b9a44a1a43 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-2-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20260202, CEPH_REF=squid, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T10:39:58.356 DEBUG:teuthology.orchestra.run.trial121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.2.service' 2026-02-13T10:39:58.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:39:58.402 INFO:tasks.cephadm.osd.2:Stopped osd.2 2026-02-13T10:39:58.402 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2026-02-13T10:39:58.403 DEBUG:teuthology.orchestra.run.trial163:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.3 2026-02-13T10:39:58.712 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:39:58 trial163 systemd[1]: Stopping Ceph osd.3 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:39:58.712 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:39:58 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3[30217]: 2026-02-13T10:39:58.499+0000 7f0c93cfc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:39:58.712 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:39:58 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3[30217]: 2026-02-13T10:39:58.499+0000 7f0c93cfc640 -1 osd.3 43 *** Got signal Terminated *** 2026-02-13T10:39:58.712 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:39:58 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3[30217]: 2026-02-13T10:39:58.499+0000 7f0c93cfc640 -1 osd.3 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T10:40:03.968 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:40:03 trial163 podman[47993]: 2026-02-13 10:40:03.93293505 +0000 UTC m=+5.446133329 container died cbffba08a22874af8bf3e9418292237e285a114400bc8412b473f355566d586e (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.label-schema.build-date=20260202, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_REF=squid, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2) 2026-02-13T10:40:03.968 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:40:03 trial163 podman[47993]: 2026-02-13 10:40:03.941564952 +0000 UTC m=+5.454763231 container remove cbffba08a22874af8bf3e9418292237e285a114400bc8412b473f355566d586e (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=squid, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260202, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, OSD_FLAVOR=default, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T10:40:03.968 INFO:journalctl@ceph.osd.3.trial163.stdout:Feb 13 10:40:03 trial163 bash[47993]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-3 2026-02-13T10:40:04.276 DEBUG:teuthology.orchestra.run.trial163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.3.service' 2026-02-13T10:40:04.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:40:04.320 INFO:tasks.cephadm.osd.3:Stopped osd.3 2026-02-13T10:40:04.320 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2026-02-13T10:40:04.320 DEBUG:teuthology.orchestra.run.trial163:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.4 2026-02-13T10:40:04.790 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:04 trial163 systemd[1]: Stopping Ceph osd.4 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:40:04.790 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:04 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:40:04.454+0000 7fda9cff8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:40:04.790 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:04 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:40:04.454+0000 7fda9cff8640 -1 osd.4 43 *** Got signal Terminated *** 2026-02-13T10:40:04.790 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:04 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:40:04.454+0000 7fda9cff8640 -1 osd.4 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T10:40:07.790 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:07 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:40:07.474+0000 7fda995ff640 -1 osd.4 43 heartbeat_check: no reply from 10.20.193.121:6806 osd.0 since back 2026-02-13T10:39:42.751904+0000 front 2026-02-13T10:39:42.751852+0000 (oldest deadline 2026-02-13T10:40:07.451771+0000) 2026-02-13T10:40:08.712 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:08 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:40:08.455+0000 7fda995ff640 -1 osd.4 43 heartbeat_check: no reply from 10.20.193.121:6806 osd.0 since back 2026-02-13T10:39:42.751904+0000 front 2026-02-13T10:39:42.751852+0000 (oldest deadline 2026-02-13T10:40:07.451771+0000) 2026-02-13T10:40:09.790 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:09 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4[36246]: 2026-02-13T10:40:09.417+0000 7fda995ff640 -1 osd.4 43 heartbeat_check: no reply from 10.20.193.121:6806 osd.0 since back 2026-02-13T10:39:42.751904+0000 front 2026-02-13T10:39:42.751852+0000 (oldest deadline 2026-02-13T10:40:07.451771+0000) 2026-02-13T10:40:10.172 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:09 trial163 podman[48261]: 2026-02-13 10:40:09.859114291 +0000 UTC m=+5.415916108 container died 749d9c037d1ca311fde2db9e44ae76fb277f56bdc6e8048ca7edd524b8d2b076 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4, org.label-schema.license=GPLv2, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, OSD_FLAVOR=default, CEPH_REF=squid, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, org.label-schema.build-date=20260202) 2026-02-13T10:40:10.173 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:09 trial163 podman[48261]: 2026-02-13 10:40:09.867536198 +0000 UTC m=+5.424338025 container remove 749d9c037d1ca311fde2db9e44ae76fb277f56bdc6e8048ca7edd524b8d2b076 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260202, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=squid, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.label-schema.license=GPLv2, OSD_FLAVOR=default, ceph=True, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2026-02-13T10:40:10.173 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:09 trial163 bash[48261]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4 2026-02-13T10:40:10.173 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:10 trial163 podman[48390]: 2026-02-13 10:40:10.014370151 +0000 UTC m=+0.015933944 container create 1233346d984ca79639d42fce0f250acdb3efdbf7f8c02310babc1102bae04298 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=squid, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.label-schema.build-date=20260202, org.label-schema.license=GPLv2) 2026-02-13T10:40:10.173 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:10 trial163 podman[48390]: 2026-02-13 10:40:10.042852997 +0000 UTC m=+0.044416800 container init 1233346d984ca79639d42fce0f250acdb3efdbf7f8c02310babc1102bae04298 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, org.label-schema.build-date=20260202, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, CEPH_REF=squid, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0) 2026-02-13T10:40:10.173 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:10 trial163 podman[48390]: 2026-02-13 10:40:10.045118967 +0000 UTC m=+0.046682760 container start 1233346d984ca79639d42fce0f250acdb3efdbf7f8c02310babc1102bae04298 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4-deactivate, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, org.label-schema.build-date=20260202, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, OSD_FLAVOR=default, CEPH_REF=squid, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, ceph=True) 2026-02-13T10:40:10.173 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:10 trial163 podman[48390]: 2026-02-13 10:40:10.045614357 +0000 UTC m=+0.047178150 container attach 1233346d984ca79639d42fce0f250acdb3efdbf7f8c02310babc1102bae04298 (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-4-deactivate, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, ceph=True, org.label-schema.build-date=20260202, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=squid, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2026-02-13T10:40:10.173 INFO:journalctl@ceph.osd.4.trial163.stdout:Feb 13 10:40:10 trial163 podman[48390]: 2026-02-13 10:40:10.007875715 +0000 UTC m=+0.009439518 image pull 4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4 2026-02-13T10:40:10.228 DEBUG:teuthology.orchestra.run.trial163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.4.service' 2026-02-13T10:40:10.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:40:10.271 INFO:tasks.cephadm.osd.4:Stopped osd.4 2026-02-13T10:40:10.271 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2026-02-13T10:40:10.271 DEBUG:teuthology.orchestra.run.trial163:> sudo systemctl stop ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.5 2026-02-13T10:40:10.540 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:10 trial163 systemd[1]: Stopping Ceph osd.5 for e9ef2631-08c7-11f1-a2b3-d404e6e7d460... 2026-02-13T10:40:10.540 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:10 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:40:10.365+0000 7f0236e35640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T10:40:10.540 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:10 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:40:10.365+0000 7f0236e35640 -1 osd.5 43 *** Got signal Terminated *** 2026-02-13T10:40:10.540 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:10 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:40:10.365+0000 7f0236e35640 -1 osd.5 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T10:40:13.040 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:12 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:40:12.632+0000 7f022f434640 -1 osd.5 43 heartbeat_check: no reply from 10.20.193.121:6806 osd.0 since back 2026-02-13T10:39:41.757354+0000 front 2026-02-13T10:39:41.757418+0000 (oldest deadline 2026-02-13T10:40:11.757543+0000) 2026-02-13T10:40:14.040 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:13 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:40:13.672+0000 7f022f434640 -1 osd.5 43 heartbeat_check: no reply from 10.20.193.121:6806 osd.0 since back 2026-02-13T10:39:41.757354+0000 front 2026-02-13T10:39:41.757418+0000 (oldest deadline 2026-02-13T10:40:11.757543+0000) 2026-02-13T10:40:15.040 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:14 trial163 ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5[42361]: 2026-02-13T10:40:14.695+0000 7f022f434640 -1 osd.5 43 heartbeat_check: no reply from 10.20.193.121:6806 osd.0 since back 2026-02-13T10:39:41.757354+0000 front 2026-02-13T10:39:41.757418+0000 (oldest deadline 2026-02-13T10:40:11.757543+0000) 2026-02-13T10:40:16.155 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:15 trial163 podman[48526]: 2026-02-13 10:40:15.859165371 +0000 UTC m=+5.505288976 container died 09d4c15cba0151d5570e29ae45566842258cec01a200ff8e9794bde2667879df (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260202, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, ceph=True, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=squid, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T10:40:16.155 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:15 trial163 podman[48526]: 2026-02-13 10:40:15.867423894 +0000 UTC m=+5.513547479 container remove 09d4c15cba0151d5570e29ae45566842258cec01a200ff8e9794bde2667879df (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, OSD_FLAVOR=default, org.label-schema.build-date=20260202, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_REF=squid, ceph=True, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-13T10:40:16.155 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:15 trial163 bash[48526]: ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5 2026-02-13T10:40:16.155 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:15 trial163 podman[48652]: 2026-02-13 10:40:15.988820298 +0000 UTC m=+0.016712861 container create de3085c7bde3f00e9c82bdc760a7b9442ae0aa18d63dd05b51bb65c7c22e5bdf (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5-deactivate, CEPH_REF=squid, org.label-schema.build-date=20260202, org.label-schema.license=GPLv2, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3) 2026-02-13T10:40:16.155 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:16 trial163 podman[48652]: 2026-02-13 10:40:16.023043879 +0000 UTC m=+0.050936382 container init de3085c7bde3f00e9c82bdc760a7b9442ae0aa18d63dd05b51bb65c7c22e5bdf (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5-deactivate, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=squid, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, org.label-schema.build-date=20260202, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T10:40:16.155 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:16 trial163 podman[48652]: 2026-02-13 10:40:16.025452364 +0000 UTC m=+0.053344867 container start de3085c7bde3f00e9c82bdc760a7b9442ae0aa18d63dd05b51bb65c7c22e5bdf (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260202, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, CEPH_REF=squid, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T10:40:16.156 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:16 trial163 podman[48652]: 2026-02-13 10:40:16.025919943 +0000 UTC m=+0.053812446 container attach de3085c7bde3f00e9c82bdc760a7b9442ae0aa18d63dd05b51bb65c7c22e5bdf (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5-deactivate, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.label-schema.license=GPLv2, CEPH_REF=squid, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260202, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3) 2026-02-13T10:40:16.156 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:16 trial163 podman[48652]: 2026-02-13 10:40:15.981820343 +0000 UTC m=+0.009712846 image pull 4cc07a41bb1966e9ffd51d6ffba181735181325200aceb7b3d1b7e810c433126 quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4 2026-02-13T10:40:16.156 INFO:journalctl@ceph.osd.5.trial163.stdout:Feb 13 10:40:16 trial163 podman[48652]: 2026-02-13 10:40:16.153527526 +0000 UTC m=+0.181420029 container died de3085c7bde3f00e9c82bdc760a7b9442ae0aa18d63dd05b51bb65c7c22e5bdf (image=quay.ceph.io/ceph-ci/ceph@sha256:20ed745b2443ef2bcde9786bc2b320e2406866236ee87c83b3f1daa2d7eecdd4, name=ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460-osd-5-deactivate, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=squid, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260202, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, OSD_FLAVOR=default, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=283431c091dafc302f5753f809d0bd1eddce0869) 2026-02-13T10:40:16.184 DEBUG:teuthology.orchestra.run.trial163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9ef2631-08c7-11f1-a2b3-d404e6e7d460@osd.5.service' 2026-02-13T10:40:16.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T10:40:16.229 INFO:tasks.cephadm.osd.5:Stopped osd.5 2026-02-13T10:40:16.229 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 --force --keep-logs 2026-02-13T10:40:16.481 INFO:teuthology.orchestra.run.trial121.stdout:Deleting cluster with fsid: e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:40:25.435 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 --force --keep-logs 2026-02-13T10:40:25.682 INFO:teuthology.orchestra.run.trial163.stdout:Deleting cluster with fsid: e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:40:30.486 DEBUG:teuthology.orchestra.run.trial121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T10:40:30.514 DEBUG:teuthology.orchestra.run.trial163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T10:40:30.542 INFO:tasks.cephadm:Archiving crash dumps... 2026-02-13T10:40:30.545 DEBUG:teuthology.misc:Transferring archived files from trial121:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841/remote/trial121/crash 2026-02-13T10:40:30.546 DEBUG:teuthology.orchestra.run.trial121:> sudo tar c -f - -C /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/crash -- . 2026-02-13T10:40:30.580 INFO:teuthology.orchestra.run.trial121.stderr:tar: /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-13T10:40:30.580 INFO:teuthology.orchestra.run.trial121.stderr:tar: Error is not recoverable: exiting now 2026-02-13T10:40:30.583 DEBUG:teuthology.misc:Transferring archived files from trial163:/var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841/remote/trial163/crash 2026-02-13T10:40:30.584 DEBUG:teuthology.orchestra.run.trial163:> sudo tar c -f - -C /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/crash -- . 2026-02-13T10:40:30.609 INFO:teuthology.orchestra.run.trial163.stderr:tar: /var/lib/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-13T10:40:30.610 INFO:teuthology.orchestra.run.trial163.stderr:tar: Error is not recoverable: exiting now 2026-02-13T10:40:30.610 INFO:tasks.cephadm:Checking cluster log for badness... 2026-02-13T10:40:30.611 DEBUG:teuthology.orchestra.run.trial121:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v 'is offline because no MDS' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v BLUESTORE_SLOW_OP_ALERT | egrep -v 'slow operation indications in BlueStore' | egrep -v 'experiencing slow operations in BlueStore' | egrep -v MGR_MODULE_ERROR | egrep -v OSD_DOWN | egrep -v 'osd.* is down' | egrep -v OSD_DOWN | egrep -v 'osd.*is down' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'osds down' | egrep -v 'osd.* is down' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'pg .* is stuck inactive' | egrep -v 'pg .* is .*degraded' | egrep -v 'pg .* is stuck peering' | head -n 1 2026-02-13T10:40:30.650 INFO:tasks.cephadm:Compressing logs... 2026-02-13T10:40:30.650 DEBUG:teuthology.orchestra.run.trial121:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T10:40:30.693 DEBUG:teuthology.orchestra.run.trial163:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T10:40:30.718 INFO:teuthology.orchestra.run.trial121.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-13T10:40:30.718 INFO:teuthology.orchestra.run.trial121.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-13T10:40:30.718 INFO:teuthology.orchestra.run.trial121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2026-02-13T10:40:30.718 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.2.log 2026-02-13T10:40:30.718 INFO:teuthology.orchestra.run.trial121.stderr:gzip/var/log/ceph/ceph-client.0.log: -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.0.log 2026-02-13T10:40:30.718 INFO:teuthology.orchestra.run.trial121.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial121.blaxtt.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mgr.trial121.tkoihk.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.0.log: /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial121.blaxtt.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.cephadm.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mgr.trial121.tkoihk.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-client.ceph-exporter.trial121.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.audit.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr: 94.0% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial121.blaxtt.log.gz 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr: 84.3% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.cephadm.log.gz 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-client.ceph-exporter.trial121.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial121.sfinrg.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr: 92.4% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-client.ceph-exporter.trial121.log.gz 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-volume.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.audit.log: /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial121.sfinrg.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.1.log 2026-02-13T10:40:30.719 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mon.trial121.log 2026-02-13T10:40:30.720 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.log 2026-02-13T10:40:30.720 INFO:teuthology.orchestra.run.trial121.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mon.trial121.log: /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.log: 91.0% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.audit.log.gz 2026-02-13T10:40:30.721 INFO:teuthology.orchestra.run.trial121.stderr: 84.0% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.log.gz 2026-02-13T10:40:30.721 INFO:teuthology.orchestra.run.trial121.stderr: 92.9% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial121.sfinrg.log.gz 2026-02-13T10:40:30.724 INFO:teuthology.orchestra.run.trial163.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-client.ceph-exporter.trial163.log 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.3.log 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-client.ceph-exporter.trial163.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.5.log 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr: 29.4% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-client.ceph-exporter.trial163.log.gz 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mgr.trial163.xfyaeo.log 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.cephadm.log 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mgr.trial163.xfyaeo.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.4.log 2026-02-13T10:40:30.725 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.audit.log 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr: 83.9% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.cephadm.log.gz 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-volume.log 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial163.mabprx.log 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr:gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mon.trial163.log 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-volume.log: /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial163.mabprx.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial163.bxtdjl.log 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mon.trial163.log: gzip -5 --verbose -- /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.log 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial163.bxtdjl.log: 91.1% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.audit.log.gz 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr: 91.6% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mgr.trial163.xfyaeo.log.gz 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr:/var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.log: 83.4% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph.log.gz 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr: 94.0% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial163.bxtdjl.log.gz 2026-02-13T10:40:30.726 INFO:teuthology.orchestra.run.trial163.stderr: 90.3% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mds.cephfs.trial163.mabprx.log.gz 2026-02-13T10:40:30.727 INFO:teuthology.orchestra.run.trial163.stderr: 94.8% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-volume.log.gz 2026-02-13T10:40:30.727 INFO:teuthology.orchestra.run.trial121.stderr: 94.8% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-volume.log.gz 2026-02-13T10:40:30.727 INFO:teuthology.orchestra.run.trial121.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-13T10:40:30.746 INFO:teuthology.orchestra.run.trial163.stderr: 93.7% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.4.log.gz 2026-02-13T10:40:30.747 INFO:teuthology.orchestra.run.trial163.stderr: 92.7% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mon.trial163.log.gz 2026-02-13T10:40:30.748 INFO:teuthology.orchestra.run.trial121.stderr: 93.8% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.2.log.gz 2026-02-13T10:40:30.748 INFO:teuthology.orchestra.run.trial121.stderr: 93.8% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.0.log.gz 2026-02-13T10:40:30.748 INFO:teuthology.orchestra.run.trial163.stderr: 93.9% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.5.log.gz 2026-02-13T10:40:30.756 INFO:teuthology.orchestra.run.trial163.stderr: 93.7% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.3.log.gz 2026-02-13T10:40:30.758 INFO:teuthology.orchestra.run.trial163.stderr: 2026-02-13T10:40:30.758 INFO:teuthology.orchestra.run.trial163.stderr:real 0m0.049s 2026-02-13T10:40:30.758 INFO:teuthology.orchestra.run.trial163.stderr:user 0m0.137s 2026-02-13T10:40:30.758 INFO:teuthology.orchestra.run.trial163.stderr:sys 0m0.023s 2026-02-13T10:40:30.765 INFO:teuthology.orchestra.run.trial121.stderr: 94.0% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-osd.1.log.gz 2026-02-13T10:40:30.769 INFO:teuthology.orchestra.run.trial121.stderr: 90.0% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mgr.trial121.tkoihk.log.gz 2026-02-13T10:40:30.814 INFO:teuthology.orchestra.run.trial121.stderr: 91.7% -- replaced with /var/log/ceph/e9ef2631-08c7-11f1-a2b3-d404e6e7d460/ceph-mon.trial121.log.gz 2026-02-13T10:40:30.815 INFO:teuthology.orchestra.run.trial121.stderr: 2026-02-13T10:40:30.815 INFO:teuthology.orchestra.run.trial121.stderr:real 0m0.109s 2026-02-13T10:40:30.815 INFO:teuthology.orchestra.run.trial121.stderr:user 0m0.278s 2026-02-13T10:40:30.815 INFO:teuthology.orchestra.run.trial121.stderr:sys 0m0.032s 2026-02-13T10:40:30.816 INFO:tasks.cephadm:Archiving logs... 2026-02-13T10:40:30.817 DEBUG:teuthology.misc:Transferring archived files from trial121:/var/log/ceph to /home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841/remote/trial121/log 2026-02-13T10:40:30.818 DEBUG:teuthology.orchestra.run.trial121:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-13T10:40:30.922 DEBUG:teuthology.misc:Transferring archived files from trial163:/var/log/ceph to /home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841/remote/trial163/log 2026-02-13T10:40:30.923 DEBUG:teuthology.orchestra.run.trial163:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-13T10:40:30.972 INFO:tasks.cephadm:Removing cluster... 2026-02-13T10:40:30.972 DEBUG:teuthology.orchestra.run.trial121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 --force 2026-02-13T10:40:31.220 INFO:teuthology.orchestra.run.trial121.stdout:Deleting cluster with fsid: e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:40:31.412 DEBUG:teuthology.orchestra.run.trial163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9ef2631-08c7-11f1-a2b3-d404e6e7d460 --force 2026-02-13T10:40:31.660 INFO:teuthology.orchestra.run.trial163.stdout:Deleting cluster with fsid: e9ef2631-08c7-11f1-a2b3-d404e6e7d460 2026-02-13T10:40:31.862 INFO:tasks.cephadm:Removing cephadm ... 2026-02-13T10:40:31.862 DEBUG:teuthology.orchestra.run.trial121:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-13T10:40:31.880 DEBUG:teuthology.orchestra.run.trial163:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-13T10:40:31.896 INFO:tasks.cephadm:Teardown complete 2026-02-13T10:40:31.896 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-13T10:40:31.906 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-13T10:40:31.906 DEBUG:teuthology.orchestra.run.trial121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T10:40:31.922 DEBUG:teuthology.orchestra.run.trial163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T10:40:32.071 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-13T10:40:32.082 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-13T10:40:32.083 DEBUG:teuthology.orchestra.run.trial121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T10:40:32.084 DEBUG:teuthology.orchestra.run.trial163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T10:40:32.099 INFO:teuthology.orchestra.run.trial121.stderr:bash: line 1: ntpq: command not found 2026-02-13T10:40:32.100 INFO:teuthology.orchestra.run.trial163.stderr:bash: line 1: ntpq: command not found 2026-02-13T10:40:32.114 INFO:teuthology.orchestra.run.trial163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T10:40:32.114 INFO:teuthology.orchestra.run.trial163.stdout:=============================================================================== 2026-02-13T10:40:32.114 INFO:teuthology.orchestra.run.trial163.stdout:^- h96-60-247-29.mdsnwi.bro> 2 6 377 27 -372us[ -372us] +/- 27ms 2026-02-13T10:40:32.114 INFO:teuthology.orchestra.run.trial163.stdout:^* 149.28.61.105.vultruserc> 3 6 377 28 -34us[ -59us] +/- 5614us 2026-02-13T10:40:32.114 INFO:teuthology.orchestra.run.trial163.stdout:^- blotch.image1tech.net 2 6 377 29 +347us[ +323us] +/- 21ms 2026-02-13T10:40:32.114 INFO:teuthology.orchestra.run.trial163.stdout:^- frumentum-zeus.infra.fru> 2 6 377 27 -1658us[-1658us] +/- 24ms 2026-02-13T10:40:32.116 INFO:teuthology.orchestra.run.trial121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T10:40:32.116 INFO:teuthology.orchestra.run.trial121.stdout:=============================================================================== 2026-02-13T10:40:32.116 INFO:teuthology.orchestra.run.trial121.stdout:^- h96-60-247-29.mdsnwi.bro> 2 6 377 27 -461us[ -461us] +/- 27ms 2026-02-13T10:40:32.116 INFO:teuthology.orchestra.run.trial121.stdout:^* 149.28.61.105.vultruserc> 3 6 377 28 -43us[ +69us] +/- 5668us 2026-02-13T10:40:32.116 INFO:teuthology.orchestra.run.trial121.stdout:^- blotch.image1tech.net 2 6 377 29 +492us[ +603us] +/- 22ms 2026-02-13T10:40:32.116 INFO:teuthology.orchestra.run.trial121.stdout:^- frumentum-zeus.infra.fru> 2 6 377 27 -1667us[-1667us] +/- 24ms 2026-02-13T10:40:32.116 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-13T10:40:32.128 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-13T10:40:32.128 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-13T10:40:32.142 DEBUG:teuthology.orchestra.run.trial121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T10:40:32.202 DEBUG:teuthology.orchestra.run.trial163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T10:40:32.246 DEBUG:teuthology.orchestra.run.trial121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-13T10:40:32.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:40:32.272 DEBUG:teuthology.orchestra.run.trial163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-13T10:40:32.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:40:32.296 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-13T10:40:32.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-13T10:40:32.324 INFO:teuthology.task.internal:Duration was 361.240002 seconds 2026-02-13T10:40:32.324 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-13T10:40:32.338 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-13T10:40:32.338 DEBUG:teuthology.orchestra.run.trial121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T10:40:32.340 DEBUG:teuthology.orchestra.run.trial163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T10:40:32.375 INFO:teuthology.orchestra.run.trial163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T10:40:32.375 INFO:teuthology.orchestra.run.trial121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T10:40:32.739 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-13T10:40:32.739 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial121.front.sepia.ceph.com 2026-02-13T10:40:32.739 DEBUG:teuthology.orchestra.run.trial121:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T10:40:32.797 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial163.front.sepia.ceph.com 2026-02-13T10:40:32.797 DEBUG:teuthology.orchestra.run.trial163:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T10:40:32.817 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-13T10:40:32.817 DEBUG:teuthology.orchestra.run.trial121:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T10:40:32.839 DEBUG:teuthology.orchestra.run.trial163:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T10:40:32.981 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-13T10:40:32.981 DEBUG:teuthology.orchestra.run.trial121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T10:40:32.982 DEBUG:teuthology.orchestra.run.trial163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T10:40:33.003 INFO:teuthology.orchestra.run.trial121.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T10:40:33.003 INFO:teuthology.orchestra.run.trial121.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T10:40:33.004 INFO:teuthology.orchestra.run.trial121.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 0.0% --verbose -- -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T10:40:33.004 INFO:teuthology.orchestra.run.trial121.stderr: 2026-02-13T10:40:33.004 INFO:teuthology.orchestra.run.trial121.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T10:40:33.006 INFO:teuthology.orchestra.run.trial163.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T10:40:33.006 INFO:teuthology.orchestra.run.trial163.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T10:40:33.006 INFO:teuthology.orchestra.run.trial163.stderr:gzip/home/ubuntu/cephtest/archive/syslog/misc.log: -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T10:40:33.006 INFO:teuthology.orchestra.run.trial163.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-13T10:40:33.006 INFO:teuthology.orchestra.run.trial163.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T10:40:33.016 INFO:teuthology.orchestra.run.trial163.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 89.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T10:40:33.028 INFO:teuthology.orchestra.run.trial121.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 90.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T10:40:33.029 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-13T10:40:33.040 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-13T10:40:33.040 DEBUG:teuthology.orchestra.run.trial121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T10:40:33.092 DEBUG:teuthology.orchestra.run.trial163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T10:40:33.117 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-13T10:40:33.128 DEBUG:teuthology.orchestra.run.trial121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T10:40:33.134 DEBUG:teuthology.orchestra.run.trial163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T10:40:33.155 INFO:teuthology.orchestra.run.trial121.stdout:kernel.core_pattern = core 2026-02-13T10:40:33.179 INFO:teuthology.orchestra.run.trial163.stdout:kernel.core_pattern = core 2026-02-13T10:40:33.197 DEBUG:teuthology.orchestra.run.trial121:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T10:40:33.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:40:33.225 DEBUG:teuthology.orchestra.run.trial163:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T10:40:33.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:40:33.251 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-13T10:40:33.263 INFO:teuthology.task.internal:Transferring archived files... 2026-02-13T10:40:33.263 DEBUG:teuthology.misc:Transferring archived files from trial121:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841/remote/trial121 2026-02-13T10:40:33.263 DEBUG:teuthology.orchestra.run.trial121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T10:40:33.296 DEBUG:teuthology.misc:Transferring archived files from trial163:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jcollin-2026-02-13_09:34:52-fs-wip-jcollin-testing-20260212.151113-tentacle-distro-default-trial/48841/remote/trial163 2026-02-13T10:40:33.296 DEBUG:teuthology.orchestra.run.trial163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T10:40:33.328 INFO:teuthology.task.internal:Removing archive directory... 2026-02-13T10:40:33.328 DEBUG:teuthology.orchestra.run.trial121:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T10:40:33.332 DEBUG:teuthology.orchestra.run.trial163:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T10:40:33.385 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-13T10:40:33.398 INFO:teuthology.task.internal:Not uploading archives. 2026-02-13T10:40:33.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-13T10:40:33.412 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-13T10:40:33.412 DEBUG:teuthology.orchestra.run.trial121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T10:40:33.414 DEBUG:teuthology.orchestra.run.trial163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T10:40:33.429 INFO:teuthology.orchestra.run.trial121.stdout: 786461 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 13 10:40 /home/ubuntu/cephtest 2026-02-13T10:40:33.429 INFO:teuthology.orchestra.run.trial121.stdout: 1195847 4 d--------- 2 ubuntu ubuntu 4096 Feb 13 10:39 /home/ubuntu/cephtest/mnt.0 2026-02-13T10:40:33.429 INFO:teuthology.orchestra.run.trial121.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2026-02-13T10:40:33.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T10:40:33.430 INFO:teuthology.orchestra.run.trial121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-13T10:40:33.430 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial121 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-13T10:40:33.431 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-13T10:40:33.442 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-13T10:40:33.457 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial121 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-13T10:40:33.459 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 361.2400016784668 failure_reason: 'Command failed on trial121 with status 1: "(cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket ''/var/run/ceph/$cluster-$name.$pid.asok'' /home/ubuntu/cephtest/mnt.0 --id 0)"' flavor: default owner: scheduled_jcollin@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-13T10:40:33.459 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T10:40:33.461 INFO:teuthology.orchestra.run.trial163.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 10:40 /home/ubuntu/cephtest 2026-02-13T10:40:33.653 INFO:teuthology.run:FAIL